CIAT
Interpreters, Iteration 9

How excited are you to run the ./script/hobbes command-line program? What if, to ensure quality control, you had to run it on every program in your ./hobbes folder and verify the output of every program for every change or addition you made?

That's exactly what acceptance testing is about. Fortunately, the process can be automated.

CIAT

CIAT (Compiler and Interpreter Acceptance Tester, pronounced "dog") is a Ruby gem that provides a rake task for interpreter acceptance tests.

A Rakefile

Since CIAT is a Ruby gem, it's best run with rake.

Here's the contents of the Rakefile:

require 'ciat'
require 'ciat/executors/java'

task :default => :ciat

desc "Run CIAT tests"
task :ciat do
  CIAT::Suite.new(
    :processors => [hobbes_interpreter],
    :folder => 'ciat',
    :report_filename => 'hobbes.html'
    ).run
end

def hobbes_interpreter
  CIAT::Executors::Java.new(
    Dir.glob('../lib/*.jar').join(':') + ":../bin",
    "edu.calvin.jdfrens.cs214.interpreters.hobbes.drivers.HobbesCLI",
    :description => "Hobbes interpreter"
    )
end

There's at least one word in here you should change.

The ciat task is set up with just one processor. (A compiler may want to compile a program and then run the generated code, so some projects may have more than one processor.)

The task will look for .ciat files in the ciat folder, and an HTML report will be generated in acceptance/temp/hobes.html. (You'll also be able to see intermediate files in acceptance/temp/ciat.)

CIAT Files

Here's a CIAT file:

simple multiplication
==== source
5*11
==== execution
55

Files like this go in acceptance/ciat.

• ==== lines are required with source and execution.
• Description of test is first. Keep it to a single sentence, but make it descriptive!
• Source should be an entire Hobbes program.
• Execution should be what gets printed out by the Hobbes interpreter. Whitespace matters!

You should be able to turn your Hobbes programs in hobbes into CIAT tests.

Running CIAT Tests

It's a rake task, so:

unix-%  rake ciat

You can open acceptance/temp/hobbes.html to see a detailed report of what your Hobbes interpreter did.

CIAT for Yourself

CIAT has been installed on our GLUW-lab computers. If you want it for your own computer, check out the README file at its GitHub repository.