DIRS ==== dbgenerator: stuff to generate a pgo.db file template: the initial template contributed by cla web: web-frontend to the pgo.db file pkgcore: pkgcore version in case you don't have one set up and use PYTHONPATH=. snakeoil: same as pkgcore, both are taken from post-0.3.1 HEAD etc: Minimal /etc/ used to tell pkgcore where the PORTDIR is, and the base make.conf/global/profile settings. todo: TODO list for the site. Look here before suggesting something. REQUIREMENTS ============ dbgenerator: ------------ - up-to-date portage tree (non-unionfs'ed as it might give weird mtimes) - Python 2.5 with sqlite or Python 2.4 with pysqlite2 - optional: pkgcore and snakeoil (both shipped) frontend: --------- CherryPy 3.0.2 or later Genshi 0.4.3 or later - Python 2.5 with sqlite or Python 2.4 with pysqlite2 it can be used both standalone or together with mod_python Database Generation Time: ========================= Jokey: ------ 3GHz P4, UDMA100 HDD, partly cluttered fs DB generation takes about 3 minutes robbat2, infra: --------------- - 2x Dual-core Opteron 2210 (4 cores total) - 4Gb RAM, cluttered FS - RAID1 WD2500JS Initial DB generation: 3 minutes Subsequent passes: 25 seconds official: --------- - dual processor, hyper-threading Intel Xeon 2.40GHz - 1Gb RAM - Clean FS - 80GB ST380011A Individual passes take about 2 minutes (IO-bound) GETTING IT GOING ================ APPROOT = the directory you checked out of Git 0) Add the following to your /etc/portage/package.keywords: =dev-python/genshi-0.4* =dev-python/cherrypy-3.0* dev-python/dbutils dev-python/formencode >=sys-apps/pkgcore-0.4.7.13 1) emerge =dev-python/genshi-0.4* =dev-python/cherrypy-3.0* dbutils formencode sys-apps/pkgcore-0.4.7.13 2) emerge pysqlite || emerge mysql-python 3) 'cp etc/database_config.py{.example,}'. Edit 'etc/database_config.py' If you want to use SQLite, set mode='sqlite' and choose a path for the file. If you want to use MySQL, edit the other settings as needed. If you are lazy, you can use a single MySQL user, but you should seperate RW and RO users for security! 4) 'cp etc/make.conf{.example,}'. Edit 'etc/make.conf' If you want a copy of the tree in a specific location, set it here. 5) ln -s "$PORTDIR" etc/portdir 6) Generate the initial database: cd $APPROOT && \ time /bin/sh ./run-update.sh 7) start CherryPy standalone: cd $APPROOT && PYTHONPATH=. python web/controller.py this should give you something like "HTTP Serving HTTP on http://0.0.0.0:8081/" 8) now point a webbrowser at http://localhost:8081/ and you should see the packages.gentoo.org site PRIVILEGE SEPARATION ==================== You can run packages2 with privileges seperated to enhance security. 1. You should have the following users: webuser (eg: apache, lighttpd) pkguser (eg: gpackages) realuser (eg: bob) 2. You should have the following groups: pkggroup (eg: gpackages) 3. Place all 3 users into the group. 4. chown -R realuser:pkggroup $APPROOT 5. chown -R pkguser:pkggroup $PORTDIR (the PORTDIR in packages2 etc/make.conf!) 6. chmod o-rwx,g+rX $APPROOT $PORTDIR TESTING BITE X