Skip to content

Run

Cubyc's Run class is the only class you need to interact with to track your experiments. Think of it as the central hub for all your needs including tracking scripts and functions, logging metrics, and querying results.

Run 

Run(params=None, tags=None, remote=None, verbose=True)

Initialize a Run instance.

PARAMETER DESCRIPTION
params

A dictionary of hyperparameters to track, where the key-value pairs are the hyperparameter names and values. Pass locals() to automatically capture local variables.

TYPE: dict DEFAULT: None

tags

Tags to associate with the experiment.

TYPE: list, tuple, or set DEFAULT: None

remote

The URL or list of URLs of the remote repositories to save the experiment to. Must be valid GitHub, GitLab, or Bitbucket URL for which you have write access.

TYPE: str, list, or set DEFAULT: None

verbose

If True, prints detailed information about the experiment run.

TYPE: bool DEFAULT: True

Example

Cubyc offers three ways to define experiment runs: you can explicitly specify the start and end of the run, utilize a context manager, or define it as a function. All three approaches are equally effective and capture the same information.

Create a Run and call its start and end methods to define the start and end of your run.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
from cubyc import Run

model = MLP(hidden_layers=2)
optimizer = Adam(lr=0.001)

run = Run(params={"model": model, "opt": optimizer}, tags=["tutorial"])
run.start()
for epoch in range(10):
    ...
    run.log({"loss": ..., "acc": ...})

model.save("models/model.pkl")
plt.savefig("plot.png")
run.end()

Use Python's with statement to define a context manager for your experiment.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
from cubyc import Run

model = MLP(hidden_layers=2)
optimizer = Adam(lr=0.001)

with Run(params={"model": model, "opt": optimizer}, tags=["tutorial"])):
    for epoch in range(10):
        ...
        run.log({"loss": ..., "acc": ...})

    model.save("models/model.pkl")
    plt.savefig("plot.png")

Define your experiment as a function and use the @run.track decorator to track it.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
from cubyc import Run

model = MLP(hidden_layers=2)
optimizer = Adam(lr=0.001)

@Run(tags=["tutorial"]))
def experiment_func(model, optimizer):
    for epoch in range(10):
        ...
        yield {"loss": ..., "acc": ...}

    model.save("model.pkl")
    plt.savefig("plot.png")
experiment_func(model=model, optimizer=optimizer)

start 

start()

Starts the experiment run.

See Also
  • Run.end : Ends an experiment run.

Examples:

Place the start and end functions around the code you want to track.

from cubyc import Run

run = Run()
run.start(params={"alpha": 7, "beta", 1e-3}, tags=["example", "start_function"])
# Your experiment code here
run.end()

end 

end()

Ends an experiment run.

See Also

log 

log(variables, **kwargs)

Logs the specified variable values to the experiment.

PARAMETER DESCRIPTION
variables

A dictionary of variable, where the key-value pairs are the variable names and values.

TYPE: dict

kwargs

Additional variables to log.

TYPE: dict DEFAULT: {}

Examples:

Call the run's log method with a dictionary containing the metrics you want to track.

from cubyc import run

run = run(remote="https://github.com/owner/project.git")

run.start(tags=["example", "log_method"])

run.log({"accuracy": 0.9, "loss": 0.1})

Alternatively, yield a dictionary containing the desired metrics if you are tracking functions.

run = run(remote="https://github.com/owner/project.git")

@run.track(tags=["example", "log_decorator"])
def my_function():
    yield {"accuracy": 0.9, "loss": 0.1}