This file contains information for graders of MS1 of HW1 (static web
server). It is written in the .md markdown format, so you can view it
in beautiful html, org or other forms. Google md markdown format for
more.
Latest update: May 15th, 2014
You need a linux/unix distribution (ideally, the course VM). When installing software, this guide assumes the apt package manager. Please make sure install traditional netcat and ApacheBench before run runtests.sh:
sudo apt-get install netcat
sudo update-alternatives --config nc
sudo apt-get install apache2-utils
There are 3 scripts (runjunit.sh, runtests.sh, runab.sh) which we
need to run to test HW1MS1. All of them take a port number as an
argument. For the last one (ApacheBench) one can use that or run the
command herself.
I have also added a script that I used myself to easily download the
submission, uncompress, compile the code and verify the README file. It
is fetch.sh.
The only thing you need to do is update the scripts depending on where you put the web root directory, that is, the one for the server to serve contents from.
Here is a mapping from the error messages which these scripts will display on the screen to the section in the grading document (the mapping below follows the structure of the grading document/grademanager). The script only prints anything if something is wrong. So, if you do not see a particular error message, you can assume it worked fine.
Documentation
- Every class/function has a proper comments -- 4
- No Documentation -- 0 points
Build procedure
- If run.sh goes through without trouble - 4 points else 0 points
README
- Check if the file contains the filenames listed
- Do a basic sanity check to make sure that any special instructions for building are incorporated in your grading.
GET support
- inconsistent results between curl and netcat -- 2
- Error: downloaded file contained NULL bytes! -- 2
- Contents of html file not preserved -- 5
- Contents of jpg file not preserved -- 5
Directory listing
- No content-length header when serving directory -- 2
- Duplicate headers when serving directory -- 2
- 3 kinds of directory listing failed is possible -- 6 (each 2 points)
HEAD support
- Response to HEAD did not end with blank line -- 1
- HEAD did not return consistent content-lengths -- 2
- HEAD not supported; headers in bH -- 2
Header checks
- Duplicate headers when serving file -- 0.25
- Duplicate headers when serving directory -- 0.25
- Did not see CR character in first line of headers -- 0.5
- No content-length header when serving file -- 0.5
- No content-length header when serving directory --0.5
- TEST 12 (modified-since) Failed -- 1
- TEST 17 (Multi-line header) Failed -- 2
HTTP/1.1
- HTTP/1.1 - did not require Host -- 0.5
- shouldn't have returned listing for HTTP/2.0 -- 0.5
- shouldn't have returned 200 OK for request without path -- 1
- Should have returned 200 OK for request with spaces after header colon -- 1
- Should not have returned 200 for request with CRASH method -- 1
HTTP/1.0
- Did not return 200 OK for HTTP/1.0 request -- 1
- HTTP/1.0 support failed, did not see test1.html -- 0.5
- HTTP/1.0 optional Host failed -- 0.5
Status codes
- TEST 5 (404) Failed -- 0.5
- TEST 13 (304) Failed -- 1
- TEST 14 (412) Failed -- 1
- Should reply 404 for missing files -- 0.5
- Should have returned 304 for request with future If-Modified-Since -- 0.5
- Should have returned 412 for request with past If-Unmodified-Since -- 0.5
- Should not have returned 412 for request with future If-Unmodified-Since -- 0.5
- Should have returned 304 for request with obscure date syntax -- 0.5
MIME types
- Incorrect header for .html file -- 0.5
- Incorrect header for .jpg file -- 1
- Incorrect header for .txt file -- 0.5
- TEST 7 (html MIME) Failed -- 0.5
- TEST 8 (jpg MIME) Failed -- 1
- TEST 9 (text MIME) Failed -- 0.5
Case sensitivity
- Should have returned 200 OK for request with hOST: header -- 3
400 if no Host:
- TEST 3 (400) Failed -- 2
CRLF/LF
- TEST 15 (CRLF/LF) Failed -- 2
Content-length
- TEST 10 (content-length) Failed -- 0.5
- length of downloaded file 'X' did not agree with content-length 'Y' -- 0.5
Absolute URLs
- Requesting absolute URL failed, got -- 1
Binary transmission
- Manually check if image is displayed -- 1
Multiline headers
- Should have returned 200 OK for request with (unknown) two-line header -- 1
Date header
- Did not see Date: header in expected format! -- 0.5
- TEST 6 (Date) failed -- 0.5
No body for HEAD
- Server returned same number of lines for GET and HEAD -- 1
Ignores unknown headers
- TEST 11 (Ignores Headers) Failed -- 0.5
- Should have returned 200 OK for request with unknown one-line header -- 0.5
Access restrictions
- TEST 2 (restriction) failed -- 3
- shouldn't have returned file for malicious URL -- 3
Invalid paths
- Use telnet to request some path which is illegal(Say GET /foo//bar HTTP/1.1)
Connection timeout
- After Sending incomplete request, waiting for timeout, if it does not print any string -- 3
Stress test After a fresh reboot, Run ApacheBench with
- 10000 requests with 1000 concurrent -- 15 (Any error after completing 9000 request is OK.)
- 10000 requests with 500 concurrent -- 10
- 1000 requests with 100 concurrent -- 5
Thread pool
- Go through code to see if they dont use java's implementation, if they do, give 0
- Go through code to see if they dont use blocking queues, if they do, give 0
- Go through code to see if they user some sort of queue variable and multiple worker threads.
Proper synchronization
- Run apache bench and see in the control panel whether all threads are currently working on some request.
Shutdown
- Go through code to see they use some sort of thread interrupts and not just system.exit()
No delay
- Try to download something really big(maybe some dvd image) and shutdown while the download is running and see if the server stopped
- Full credit only if they passed above test case
We've tried to create a reverse mapping where you can directly see how many points to cut if a jUnit test fails. Please cross reference if you change your grading scheme.
Feel free to [drop me a line] (mailto:nvas@seas.upenn.edu) in case you need any help.