Prepare a Harbor task

A Harbor task is a folder that holds a task.toml, an instruction, an environment, a reference solution and a verifier. Before you submit, check each item on this page against your task. Each item names the rule of the intake standard that it answers.

The task folderCopy Anchor Link

A task that uses a Dockerfile looks like this:

my-task/
  instruction.md
  task.toml
  environment/
    Dockerfile
  solution/
    solve.sh
  tests/
    test.sh

Horizon Audit finds a task by its task.toml. Every folder of your archive that holds a task.toml is one task.

The checklistCopy Anchor Link

Files that must existCopy Anchor Link

• instruction.md exists and has text. An empty file does not count. (IS-1)
• tests/test.sh exists. This is the verifier, and the verifier is the grader. (IS-2)
• task.toml parses. (IS-3)
• The task declares an environment in one of three forms: environment/Dockerfile, environment/docker-compose.yaml, or docker_image under [environment] in task.toml. A compose file under any other name does not count. (IS-4)
• solution/solve.sh exists, and runs something. A script that holds only comments is a placeholder, not a solution. (IS-5)

A Windows task is one whose os under [environment] in task.toml is "windows". For a Windows task the verifier is tests/test.bat and the reference solution is solution/solve.bat.

The environmentCopy Anchor Link

• The environment builds today, from what the task declares. A dependency that has changed, or a download that is gone, stops the build. (IS-6)
• A base image named by tag is accepted. You do not need to name it by digest.
• environment/docker-compose.yaml, when the task has one, uses only settings that keep the task inside its sandbox. It does not ask for access to the host. (IS-8)
• cpus, memory_mb and storage_mb under [environment] are within the limits of the audit sandbox. (IS-9)

If task.toml declares a separate verifier environment under [verifier.environment], then IS-4, IS-6 and IS-9 apply to that environment as well.

SecretsCopy Anchor Link

• Every key or outside account the task needs is declared in task.toml as ${NAME}, and a value is supplied for each name. (IS-7)

NamesCopy Anchor Link

• Each task of one archive has its own name in task.toml. Two task folders that give the same name stop the whole archive from being read. A task with no name goes by its path.

What the checklist does not coverCopy Anchor Link

The intake standard asks that the reference solution is present. It does not ask that the solution works. A solution that is present and fails gives a flawed verdict, not a request. So run your reference solution against your own verifier before you submit. It must score full reward.

The verifier must write one reward that the audit can read. If it writes several rewards, one of them must be named reward.

A pinned base image is not required. A network policy is not an intake rule. The task's network setting is never weakened to make the task run.

Before you submitCopy Anchor Link

A task is pinned to a digest when it is read. The digest is a hash of the task's files, their paths and their contents. A change of one byte gives a new digest. A collection name is used once, and a changed task is sent again under a new name. So finish the task first, and then submit it. Submit tasks is the next step.