TODO 0.9: Tools: 1: A commit script to run tests, etc. Can also record ohcount stats. 2: Testutil- maybe- if no output file given, print to stdout, remove the human-readable option. Docs: 1: Turn method/class/fuction names in tutorials to links to the API doc. Also dates of generation on tutorials - if not through txt2tags, through python script 2: INISection::GetName/GetNameSTL isn't mentioned in any tutorial. Perhaps add it to Getting Started. Or at least add links to INISection API reference (maybe a Read More section) 3: Split features on the site, in README and on Launchpad to categories, such as speed, debugging, etc. With nice HTML on the site so it's clearly differentiable. 4: Review tutorials, simplify them, make them more readable, improve them in general. Use bold/italic, paragraphs to improve readability. Simplify sentences where possible by removing unnecessary words. Maybe try to use open text summarizer for help. To the getting started tutorial, add a better intro - with an about section , a link to download, etc. getting started also needs to be separated into more subsections (add sub headers to the code itself part) - maybe the same with other tutorials. paragraphs - 3-5 sentences. Use tables when/if needed. Getting started should also mention asserts, not only warnings, should be at most ~30 mins and should say at the beginning and/or in title that we're going to show all functionality needed to use MiniINI (i.e. stuff like ReadXXX) 5: Proofread API docs to remove stupid copypaste bugs and improve it. Learn more about doxygen tags, and maybe use some more of them. Also make sure everything has basic description in API docs. Specify where we require arguments to have certain values (e.g. output buffers have to be allocated) 6: In the iteration tutorial, mention that GetSection("") won't return NULL unless we iterate outside the file and that there's an assert for that. 7: Change all comment, help and doc strings to normal dot ended sentences TODO 0.10+: Tools: 1: Projects for various IDEs and make alternatives. Move default build system to something else than make. Probably SCons. Then create a benchmark-extra target. 2: Tester: provide tests for situations such as NULL buffer passed to LoadBuffer, non-INI, nonexistent file passed to OpenFile etc. We also need tests designed specifically to test Allocator. Maybe even separate testgroup that doesn't even use INIFiles/Sections, just Allocator directly. 3: Tester: Verbosity options: Normal: no MiniINI warnings, minimal info Verbose: no MiniINI warnings, detailed info More verbose: MiniINI warnings, detailed info 4: Makefile: Benchmark, tester should be non-essential. I.e. if making any of them fails, emit a warning but make everything else. 5: A printini tool (binary), that prints all contents of MiniINI, with chars like \n or EOL emphasized. 6: Benchmark (no need for bench-extra as it won't cause slowdown') - memory usage calculations (i.e. sizeof(INIFile) + sizeof(Allocator) + size of Allocator blocks + INIFile length * (sizeof(INISection *) + sizeof(INISection)) + (for every INISection) INISection length * sizeof(c *) + size of all globals and statics. This can't measure bookkeeping bytes, but can still provide an alternative to Massif measurements. Use measured data to consider minimizing memory usage by decreasing size of INISection members, global buffers, both in debug and optimized, measure cache performance after any optimizations. - names and tags are stored in allocator, so no need to count them twice 7: Use a script to replace contents the rulesmd.ini test file by randomly generated contents with equally long strings 8: Scripts- try to refactor all XXXGroup functionality to few/one class/es (maybe put that into a module?) - at least create a base class if possible Error checking/Debugging: 1: Modify parseint and maybe other functions to make sure they can work even on machines where int is 16 or 64 bit. Probably use #ifdef to do that. 2: Make sure there never are two tags or sections with the same name (+ warnings). Or at least have the warnings and mention this in documentation. This will have to wait until current Insert based sorting is replaced as optimized builds break this functionality. Interface: 1: Writing functionality - AddTag, AddSection, DeleteTag, DeleteSection, SetTag Deleting should be the easiest part, so let's do that first. Note that deleting sections will need a specific INISection destructor. The priority should be to get _working_ writing first, fast later. So don't worry even if every single AddXXX, DeleteXXX, SetXXX results in writing to file. Maybe SetTag and AddTag should be the same method that adds a new tag if it doesn't exist. Speed: 1+: Sorting idea: When initializing static INISection data, also create arrays or c* for every char (i.e. 255 of them). These will be used in bucket sorting like so: When section loading, every tag goes to bucket with its starting char, and is sorted using the standard insert method there. When we're finished, concatenate the buckets to the Tags array. Implement this and benchmark. Especially benchmark this on rulesmd2.ini and on arrays (where almost all elems will go to the same bucket so overhead will be greatest) 2+: Instead of 4; implement MSD radix sort. Both 4; and 5; have to be benchmarked on huge array input which should be slow for the current sort. 3: ReadStrings code as macro in ReadInts/Floats/Bools might be useful as there would be no need to create a buffer of pointers to strings, they could be used directly. This would get us at best about 4%, usually about 1.5%. However, it would require removing ParseInts, ParseFloats, ParseBools, and make code much more of a mess. Fortunately, most of time wasted is spent on new/delete, which means: To Allocator, add a special on-demand block, that doesn't get deleted, but is small and only capable of holding a single buffer at a time. This removes reallocating complexity as, since there are no other buffers in the block, we can reallocate freely when we ask for memory. So create such a block and use it for methods that only allocate memory once, in a small buffer. This block should never contract, unless it gets over some size that's considered too much (so when we're loading multitags with 10000 values, the memory gets freed right away) - this "too much" size should be relative to file size (5%?), but at least 64 bytes (usual cache line size) However, this might not be needed. At first, just use the normal allocator. Then measure any increases in memory use (as each new buffer gets deleted right after it's allocated, allocator should quickly delete any new blocks) Note that this will require changes to allocator's NewSpace, DeleteSpace methods. Void * might be used as an argument, but templates would probably be better. Implement, benchmark. 4: Try to improve branch prediction in most used functions Try using __builtin_expect + -freorder-blocks or -O2 . Also try to decrease cache misses by manual prefetching. Have to benchmark that with bench-extra to get time, not instruction count 5: Optimized string operations (strlen, etc.) as described in strchr.com articles. Possibly optional code specific for SSE 4.2 . Optimizations based on Intel optimization manual. 6: Try using attribute __const__ for pure functions 7: Optimizations specifically targeting reading small files (<32kiB, as well as <1kiB) -especially Allocator could handle smaller files better (allocate less blocks to waste less memory) Currently, memory usage won't go below 4135 bytes even with an empty file. It might be possible to halve that. Refactoring: Docs: 1: What MiniINI loads and how (for instance, empty section headers are ignored) 2: An article about MiniINI parsing process (After sorting gets rewritten) Web: TODO 1.0+ Tools: 1-: Reintroduce profile guided optimization, include in release Note: Library PGO can use gca files generated by profiling run (although we might have to set the not-exactly-same-source error to warning) 2?: Better INISection array benchmark: should search for arrays in a section by itself (need to find 2 tags starting with the same characters - start by comparing first char, then another, etc, if we get to a number in multiple tags starting by the same letter, find 1 and 2, if done, we can assume we found the array.) 3: .deb packaging script 4?: Maybe a simple GUI for test.py (whether to benchmark or compare, which inifiles to use, etc) 5: testutil.py: support for gprof options for memcheck, cachegrind, callgrind, massif/ms_print maximum, average heap, stack and bookkeeping bytes from massif out file rename to a more descriptive script name (other scripts might need that as well) 6-: dynamic linking 7: testutil.py: More detailed stats with bench-extra and possibly run time tests - i.e. mean, median, average error, measurement errors 8?: Test.py - possibly- stats about how tag count, etc. affect execution time, memory use etc. (might need a separate tool) - e.g. does this value increase execution time linearly? etc. Error checking/debugging: 1: warning when whitespace is ignored in a header warning when [ or ] is detected in a val warning when there's only ] in line warning when there are multiple sections/tags with same name warning when there is [ and ] in wrong order warning when there is [ and/or ] with name=value separator in the same line warning when there are characters after ] in a line warning when there is [ after [ warning when two-more tags/sections with same name and ignore all but the first one warning when header is empty 2: Thread safety Interface: 1+: A plain C wrapper, Vala wrapper, maybe a Python wrapper as well 2: A (template?) generic Read method for serialization, that will construct an object as long as it has a string constructor. E.g. a template method that will execute ctor of given type on the referenced variable to write to. Maybe a corresponding write method as well. This might require more intelligent whitespace handling - either using double quotes or removing spaces around tag names and values - the latter isn't impossible, one possible implementation is to keep a space counter variable, and if we hit a non-space char after a space sequence., we add that many spaces. If we hit the end of line, we don't add them. In order to prevent an if check every character if we've had a space sequence last character we could enter a separate loop with the first space that will exit at first non-space character. If done right this could mean overhead only when spaces exist. Also offer detailed stats, i.e. which items take how much memory. Speed: Refactoring: Docs: 1+: Write a simple example game using MiniINI (Tetris, Asteroids, Arkanoid clone or something along those lines) Web: 1?: Wiki(once needed, _if_ ever needed)