Memory Optimizations to Reduce CPU Costs

Padding in the post refers to memory alignment.

If we had an "array of structs" instead of "struct of arrays" it would be: string(8) + long(8) + int(4) + padding(4) = 24 bytes

Apparently you're not interested in thinking either, which is another thing I've noticed with many developers these days...

The sibling comment provided a good hint already. All you need to store are some file offsets, amounting to a few dozen bytes.

If the “Task” is outputting the JSON for terms to a file, it can be streamed one row at a time - with memory reused after each row is read and the output written. That could be done with a few KB of program space assuming you’re parsing the CSV and outputting the JSON manually instead of using a larger library.

The problem isn’t well constrained because it seems to imply that for some reason it needs to be all accessible in memory, doesn’t specify the cardinality of terms, doesn’t specify whether Get(i) is used in a way that requires that particular interface for accessing a row by number.

If I were to do it, I’d just parse a Page at a time and update a metadata index saying Page P contains entries starting at N. The output file could be memmapped and only the metadata loaded, allowing directly index into the correct Page which could be quickly scanned for a record, and would maybe use 1-2MB of RAM for metadata and whatever Pages are actually being touched.

But like I said the problem is not well constrained enough for even a solution like that to be optimal, since it would suffer from full dataset sequential or random access, as opposed to hot Pages and a long tail.

/shrug specs matter if you’re in the optimization phase

The cost of a string array is paid on every GC phase. That array may/contains references so the gc has to check each element every time to check if anything changed. An int array cannot contain references so it can be skipped.

edit: There are tricks to not traverse a compound object every time, but assume that at least one of the 80M objects in that giant array gets modified in between GC activations.

Even without a GC actual strings are potentially expensive because each of them is a heap allocation, if you have a small string optimisation you avoid this for small strings (e.g. popular C++ standard library string types can have up to 22 bytes of SSO, the Rust CompactString has 24 bytes) but I wouldn't expect a GC language to have the SSO.