Pcforever

From ProgClub
Revision as of 14:01, 6 June 2014 by John (talk | contribs) (→‎Technical specification: Adding link/gzip...)
Jump to: navigation, search

Pcforever is the ProgClub caching HTTP gateway software. That's the software that proxies HTTP requests and caches the results forever. For other projects see projects.

Status

Under way. Not released yet, there's stuff TODO.

Administration

Contributors

Members who have contributed to this project. Newest on top.

All contributors have agreed to the terms of the Contributor License Agreement. This excludes any upstream contributors who tend to have different administrative frameworks.

Copyright

Copyright 2014, Contributors.

License

Licensed under the New BSD license.

Resources

Downloads

There are no downloads for this project. Get the software from svn.

Source code

The repository can be browsed online:

https://www.progclub.org/pcrepo/pcforever/branches

The latest stable released version of the code is available from:

https://www.progclub.org/svn/pcrepo/pcforever/tags/latest

Or if you want the latest version for development purposes:

https://www.progclub.org/svn/pcrepo/pcforever/branches

Links

N/A.

Specifications

Functional specification

The functional specification describes what the project does.

The software intercepts HTTP requests to a base path, e.g. "/theme". Anything under the base path, e.g. "/theme/all.css" gets checked to see if it's in the cache. If it is in the cache then the software returns its contents (and mime-type) and completes. If it is not in the cache the software proxies the request to another server, e.g. "https://www.example.com/theme/all.css". The results of the proxy request are cached so that next time that resource is requested it is served from the software's local cache. The idea is to have a system that caches upstream HTTP data forever.

Why is that useful? I'm glad you asked! :)

I'm studying at the moment and have temporary access to my university web-server. Access will be revoked when the course completes. When I save pages (in Firefox with File -> Save As) served from the uni server there are some resources that aren't included in the saved page. For example the CSS file uses the 'url' directive and uses root-relative paths to things like /theme/example.png. I need some software to intercept requests for such content (which at the moment just 404) and then continue to provide that content even when I no longer have access to the uni server. Makes sense?

Technical specification

The technical specification describes how the project works.

There is a front-controller controller.php which receives HTTP requests. It is configured for a base path in Apache with something like this:

Alias /theme /var/www/pcforever/0.1/web/controller.php

The CACHE constant is defined as something like:

/var/www/pcforever/0.1/dat

The UPSTREAM constant is defined as something like:

https://www.example.com/theme

The CACHE format is:

/cache/xx/yyyy/{link,mime,data,gzip}

Where:

  • xx = first two characters of the SHA1 hash of the resource
  • yyyy = all 20 characters of the SHA1 hash of the resource
  • link = a text file with the upstream URL on the first line
  • mime = a text file with the content-type on the first line
  • data = a binary file with the content for the request
  • gzip = the gzipped content of the data file

Notes

Notes for implementers

If you are interested in incorporating this software into your project, here's what you need to know:

Copy-and-paste your way to victory!

Notes for developers

If you're looking to set up a development environment for this project here's what you need to know:

Checkout the software to some place that Apache can serve it. In my case I checkout branches to ~/workspace/pcforever. I have Apache using /home/jj5/workspace as its root directory, so I can hit the front-controller via Apache at:

http://localhost/pcforever/0.1/web/controller.php/example/content.png

Tasks

TODO

Things to do, in rough order of priority:

  • Proxy HTTP requests
  • Capture HTTP response (data and mime-type)
  • Store HTTP response (data, mime-type and gzipped data)
  • Serve from cache if possible (using gzip if possible)

Done

Stuff that's done. Latest stuff on top.

  • JE 2014-06-06: created project page
  • JE 2014-06-06: created the project in svn