Adding blobs binary search index to PBF

General talk
7

INT32 size_of_nodes_table is just number of blobs, aka. how many tiles you should read INT64 FirstNodeId, INT64 FileOffset, which is currently 30,711 for nodes.

On topic of GOL, I saw it before, I even played with it, but lack of updating with .osc was blocker for adoption.

But even for GOL conversion from .pbf it would probably beneficial from this, since it would allow to immediately start multithreaded extraction of pbf.

Regarding performance vs. tool like GOL, it will never reach such performance but for example executing code like this:

var sw = Stopwatch.StartNew();
var index = PbfIndexBuilder.LoadIndex(path);
foreach (var node in NodesParser.LoadNodes([
        2389124702,
        5398559205,
        6453678807,
        6578673562,
        7004290281,
        7294954845,
        7856041912,
        7964450430,
        8983947116,
        9376228339
], index))
{
    Console.WriteLine(node.Tags);
}
Console.WriteLine(sw.Elapsed);

Takes 100 milliseconds, which is acceptable for most use cases…

@Jochen_Topf I build this index first time I see .pbf without .pbf.index next to it, it takes like 80seconds, because 1GB/s limit of hard drive, so it’s not end of the world, but would be nice if running tools like osmium on freshly downloaded downloaded planet.pbf that shows random node content under 1second or something like that.
Regarding, increasing size of blobs, yes it would make this a bit less effective, but I believe it would still be much faster then without index.