Swish-e Wishlist

From Swishewiki

• multibyte support !

• Accept a command line option to specify anything that can be specified in a swish-e configuration file. Perhaps like

swish-e -i mydatadir -f myindex.index -O'IndexOnly .html'

• runtime option to NOT normalize/scale ranks to 1000

• make index files platform independent (specify bit size and layout of data in indexes; perhaps store ints in 'network order' and use typedefs like UINT32 etc)

• update Gentoo Portage of swish-e (Ed- Done as of 7/25/05)

• documentation and testing of PHP interface

• help output for 'spider.pl --help' or 'spider.pl -h' -- perhaps it could output the perldoc of itself?

• Swish-e test suite (In Progress: see http://joshr.com/src/Swishetest/ )

• Regular Expression Buzzwords
  • For example, the ability for a phrase like "ERR-1234" to get indexed as "err-1234" instead of "err" and "1234".
  • Possible configuration file directive: BuzzWordsMatch /ERR\-\d+/
  • For more info, see http://swish-e.org/Discussion/archive/2004-03/7186.html

• Option to allow an optional "reversed-word" index or meta to support searching for word endings (and perhaps fragments).
  • (This could also include some sort of higher-level integration, ie with the query parser. For now, people can hack around this by creating metanames with words reversed, and handle search issues with a custom query interface. --Joshr 10:14, 23 Dec 2004 (EST))

• Ability to detect indexes from earlier versions of swish-e, at least moving forward; then swish-e could report 'index from earlier version' instead of 'empty index'.

• Show which input file/Path-Name errors like "input conversion failed due to input errorBytes: 0x87 0x40 0x92 0x6E" come from.
• (See also the latest version of the the swish-e TODO list)

• Put pdf of docs on website.

• Add support to prevent traversing URL's outside the base URL in spider.pl::process_link() ie: return unless ( $uri =~ /^$server->{base_url}/ ); (Ed: this can be done with the test_url() function: see http://swish-e.org/current/docs/spider.html#item_test_url )

• Allow IndexDir to look at a scheme and decide what indexing method to use (fs, prog or html)
  • Possible syntax: IndexDir prog:///path/to/program file:///path/to/dir
  • For more info, see http://swish-e.org/Discussion/archive/2004-04/7370.html

• Extend the -k option with information, how often the keyword is used in the index and/or in how many documents it is included. I guess the information is already somewhere in the index file, there's just no way to access it, at least none I know of

• some statistic information about the index