19
Definition of a `problem`:
21
A `problem` consists of the following components:
23
+ A *problem-code*, for internal use.
24
+ A human-readable *problem-name*.
25
+ An *input-dir* [in absolute or relative form?].
27
+ A *time-limit* (default: 2.0 seconds).
28
+ A *mem-limit* (default: 2.0 megabytes)
30
Definition of a `solution`:
32
A `solution` consists of the following components:
34
+ The *problem-code* indicating which problem to test.
35
+ The *solution-file* in absolute path.
36
+ The *solution-output* in adsolute path.
37
+ The *language* which the solution uses (keyword).
40
Definition of a `test-case`:
42
A `test-case` consists of the following components:
46
+ *from-test* and *to-test* numbers indicating the range of tests.
50
* Each problem has a time limit for all tests.
52
* Each problem has a memory limit for all tests.
54
* Each problem doesn't need to know about its test cases. E.g. test
55
cases are stored in flat files and are automatically importing when
58
* The test set of a problem is based on flat files. Hence, there's no
59
need to define a test information in the problem class.
61
* Ignore the unexisted test when testing.
63
* Each test set for a problem has to be packed into a .tar.bz2 file
64
and the tests has to be numbered consecutively.
66
- An input file has to be of the form <test-numeber>.in
67
- An output file has to be of the form <test-number>.out
68
- An answer file has to be of the form <test-number>.ans
70
* The `input_directory` passed to `run-test.sh` must end with a slash
73
* The test output for each test must be of the form
74
`<solution-id>_<test-num>.out`.
81
+ Correct intepreter paths in `conf-interpreter.lisp`.
82
+ Correct problem database in `conf-problems.lisp`.
83
+ Correct the solution database in `conf-solutions.lisp`.
84
+ Custom the listening port in `conf-network.lisp`.
89
The testing process is described as follow:
91
* Initially, the judge system will read the configuration from 4
94
- `conf-data-structures.lisp`
95
- `conf-interpreter.lisp`
96
- `conf-problems.lisp`
97
- `conf-solutions.lisp`
100
* It then setups the appropriate components for testing, including:
102
- Network: an IP and a listened port.
105
* Then, it initializes the `*solution-queue*` and `*result-queue*`
106
variables, set them to `nil`. Putting the system into an idle
107
state, ready for action.
109
* Whenever a problem is submitted, it will:
111
- Add the problem to `*problem-list*`.
112
- Add the problem to the database.
114
* Whenever a solution is submitted, it will:
116
- Be put into the `*solution-queue*`.
118
- For each solution in `*solution-queue*`, a process which does the
119
following things are created:
121
* Compile the solution, using the corresponding helper:
122
`*compile-<language>.sh*`
124
+ For compiled languages, the solution will be compiled as normal.
125
+ For interpreted languages, the solution wil be compiled into
126
bytecode with basic optimization options.
128
* Put the results of the compilation process into the appropriate
129
testing directory with the appropriate permission, default
132
* For each test, run the solution and immediately check for
134
+ *Memory limit*, halt the testing if a test failed.
135
+ *Time limit*, halt the testing if a test failed.
137
If no test failed for mem-limit of time-limit, the results of
138
every test case are combined into a list whose element is of the
139
form `(list test-number result)`. `result < 0` indicates that
140
there are some problems running the test:
142
+ `result = -1` time limit exceeded.
143
+ `result = -2` memory limit exceeded.
144
+ `result = 0` zero point.
146
If `result <= 0`, no more test case in the test set would be
149
* Report the result of the solution by putting it into the
150
`*result-queue*`. `*result-queue*` is then read via network of
151
named pipe, each time it is read, it is popped (proprosed data
152
structure: *multicast stream*)
157
* Run each test case in a test set concurrently.
159
* Supporting problems which require more than
160
output-comparison-checking method.
162
* Proposed data structure for `*result-queue*` and `*solution-queue*`:
167
- Console (probably with cl-charms)
168
- GUI (probably with cl-gtk2)
added
removed
Notes
