• Frank Lichtenheld's avatar
    Complete rewrite of create_index_pages · c6fdd0f2
    Frank Lichtenheld authored
    The old version was very hard on all of CPU, memory,
    and I/O. The new version tries to at least lessen the
    burden on memory (and to a lesser extend on CPU), in the
    hopes that this will indirectly (by lesser swapping) also
    reduce the stress on I/O.
    In the old version we first collected all the information
    about all packages and then wrote the index files one after
    one with TT.
    The newer version tries to write all of the index files while
    collecting the package information, therefor eliminating the
    need to store any information longer than needed. Because the
    amount of files we can open simulaniously is limited though,
    we can't write all the index files at once. Instead we write
    one index files for all languages, and in a second step then
    use these files to write one file per language. I tried to
    use the "slice" program for that, but it also computes all
    the results in memory, while I wanted to have a concurrent
    approach here as well.
    Since we only use a very limited subset of the features of slice
    I wrote a simple replacement that supports the bare minimum of
    what we need with the price that we need to be a bit careful
    about the format of the slice source files.
    I also don't use TT for the package entries anymore since here
    as well we only used a very limited set of features and I wanted
    to make this script as fast as possible (without having to rewrite
    it in C ;). The header and footer still use TT since this part isn't
    speed critical and we can reuse the templates we have for the dynamic
Last commit
Last update
index_foot.tmpl Loading commit data...
index_head.tmpl Loading commit data...