Skip to content

Repository files navigation

AZZMOS PROJECT
==============

VERSION 0.1

Azzmos is WIP project that attempts to build a multi-threaded web crawler that can use different
index algoritms.  The aim of the project is to provide three programs on completion:

These are:
   azzmos  - web crawler.
   azidx   - indexer
   azctl   - controller function.

The program requires a running and multi-threaded version of postgresQL database.

COMPILING POSTGRESQL
====================
postgresql will need to be compiled in the following way:

./configure --enable-thread-safety
make 
make install

In the postgresql.conf file the following should be added:

client_encoding = iso-8859-1

INSTALLATION INSTRUCTIONS
=========================
Ensure that psotgresql is installed and running.

Modify the ddl/install.sh script.

Create a build directory 'mkdir build'

Change directory to build,  'cd build'

Use the following commands:

../configure
make 
make check
make install



PREREQUISITS
============
To install azzmos the following is required:
    PostgresQL compiled to run in multi-threaded mode (v 8.2.16)
    cURL development headers
    PCRE library and headers



About

general purpose web crawler for UNIX systems using PosgresQL backemd

Resources

Stars

1 star

Watchers

1 watching

Forks

Releases

Packages

Contributors

Languages