{ "cells": [ { "cell_type": "markdown", "id": "956c4a0f", "metadata": {}, "source": [ "# Durable sequential tasks\n", "\n", "`TaskRunner` runs tasks one at a time in the PythonHere process and persists\n", "their inputs, results, errors, and retry state. Work continues if Jupyter\n", "disconnects. After PythonHere restarts, completed tasks remain complete and an\n", "interrupted task becomes runnable again.\n", "\n", "## Connect to PythonHere" ] }, { "cell_type": "code", "execution_count": 1, "id": "bc2cc113", "metadata": { "execution": { "iopub.execute_input": "2026-10-15T11:08:48.985932Z", "iopub.status.busy": "2026-10-15T11:08:48.985534Z", "iopub.status.idle": "2026-10-15T11:08:49.769341Z", "shell.execute_reply": "2026-10-15T11:08:49.768311Z" } }, "outputs": [], "source": [ "%load_ext pythonhere\n", "%connect-there" ] }, { "cell_type": "markdown", "id": "63bbfb86", "metadata": {}, "source": [ "## Submit and start work\n", "\n", "Every task needs a unique nonempty string `id`. Tasks and returned results must\n", "contain plain JSON-compatible values. If `\"squares-example\"` already exists\n", "from an earlier attempt, use a new name or delete the old run with the\n", "[cleanup cell](#clean-up) at the end." ] }, { "cell_type": "code", "execution_count": 2, "id": "b27f1e4b", "metadata": { "execution": { "iopub.execute_input": "2026-10-15T11:08:49.775109Z", "iopub.status.busy": "2026-10-15T11:08:49.774151Z", "iopub.status.idle": "2026-10-15T11:08:50.391832Z", "shell.execute_reply": "2026-10-15T11:08:50.390582Z" } }, "outputs": [], "source": [ "%%there\n", "import time\n", "\n", "from tasks_here import TaskRunner\n", "\n", "\n", "def execute_task(task, context):\n", " time.sleep(1)\n", " if task[\"value\"] == 5:\n", " raise ValueError(\"Intentional example failure for value 5\")\n", " return {\n", " \"value\": task[\"value\"],\n", " \"square\": task[\"value\"] ** 2,\n", " }\n", "\n", "\n", "runner = TaskRunner.create(\"squares-example\", max_attempts=3)\n", "runner.add([{\"id\": str(value), \"value\": value} for value in range(1, 11)])\n", "runner.start(execute_task)" ] }, { "cell_type": "markdown", "id": "304ada1b", "metadata": {}, "source": [ "`start()` returns promptly while tasks run one at a time in a background thread.\n", "The task for value `5` deliberately fails. With `max_attempts=3`, the runner\n", "tries it three times before marking it failed; the other nine tasks still run.\n", "Including retries, this example takes about twelve seconds. Tasks continue\n", "running between notebook cells, so the run may already be completed when you\n", "inspect it.\n", "\n", "## Inspect progress and results" ] }, { "cell_type": "code", "execution_count": 3, "id": "3ad4b287", "metadata": { "execution": { "iopub.execute_input": "2026-10-15T11:08:50.397417Z", "iopub.status.busy": "2026-10-15T11:08:50.396781Z", "iopub.status.idle": "2026-10-15T11:08:50.468951Z", "shell.execute_reply": "2026-10-15T11:08:50.468201Z" } }, "outputs": [ { "data": { "text/plain": [ "{'name': 'squares-example',\n", " 'state': 'running',\n", " 'total': 10,\n", " 'pending': 9,\n", " 'running': 1,\n", " 'completed': 0,\n", " 'failed': 0,\n", " 'progress': 0.0,\n", " 'current': '1',\n", " 'pause_requested': False,\n", " 'max_attempts': 3,\n", " 'exclusive': True,\n", " 'created_at': '2026-10-15T11:08:50.392116+00:00',\n", " 'updated_at': '2026-10-15T11:08:50.498292+00:00',\n", " 'metadata': {},\n", " 'worker_error': None,\n", " 'path': '/data/user/0/me.herethere.pythonhere/files/task-runs/squares-example'}" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "status = %there get runner.status()\n", "status" ] }, { "cell_type": "markdown", "id": "44c4481e", "metadata": {}, "source": [ "You can inspect results before the whole run finishes:" ] }, { "cell_type": "code", "execution_count": 4, "id": "34b65777", "metadata": { "execution": { "iopub.execute_input": "2026-10-15T11:08:50.472865Z", "iopub.status.busy": "2026-10-15T11:08:50.472536Z", "iopub.status.idle": "2026-10-15T11:08:50.540362Z", "shell.execute_reply": "2026-10-15T11:08:50.539775Z" }, "tags": [ "hide-output" ] }, "outputs": [ { "data": { "text/plain": [ "[{'id': '1',\n", " 'status': 'running',\n", " 'task': {'id': '1', 'value': 1},\n", " 'result': None,\n", " 'result_present': False,\n", " 'attempt_count': 1,\n", " 'failure_count': 0,\n", " 'attempts': [{'number': 1,\n", " 'started_at': '2026-10-15T11:08:50.495452+00:00',\n", " 'finished_at': None,\n", " 'duration': None,\n", " 'error': None}],\n", " 'duration': None,\n", " 'total_duration': 0,\n", " 'error': None,\n", " 'created_at': '2026-10-15T11:08:50.455575+00:00',\n", " 'completed_at': None},\n", " {'id': '2',\n", " 'status': 'pending',\n", " 'task': {'id': '2', 'value': 2},\n", " 'result': None,\n", " 'result_present': False,\n", " 'attempt_count': 0,\n", " 'failure_count': 0,\n", " 'attempts': [],\n", " 'duration': None,\n", " 'total_duration': 0,\n", " 'error': None,\n", " 'created_at': '2026-10-15T11:08:50.455746+00:00',\n", " 'completed_at': None},\n", " {'id': '3',\n", " 'status': 'pending',\n", " 'task': {'id': '3', 'value': 3},\n", " 'result': None,\n", " 'result_present': False,\n", " 'attempt_count': 0,\n", " 'failure_count': 0,\n", " 'attempts': [],\n", " 'duration': None,\n", " 'total_duration': 0,\n", " 'error': None,\n", " 'created_at': '2026-10-15T11:08:50.455842+00:00',\n", " 'completed_at': None},\n", " {'id': '4',\n", " 'status': 'pending',\n", " 'task': {'id': '4', 'value': 4},\n", " 'result': None,\n", " 'result_present': False,\n", " 'attempt_count': 0,\n", " 'failure_count': 0,\n", " 'attempts': [],\n", " 'duration': None,\n", " 'total_duration': 0,\n", " 'error': None,\n", " 'created_at': '2026-10-15T11:08:50.455924+00:00',\n", " 'completed_at': None},\n", " {'id': '5',\n", " 'status': 'pending',\n", " 'task': {'id': '5', 'value': 5},\n", " 'result': None,\n", " 'result_present': False,\n", " 'attempt_count': 0,\n", " 'failure_count': 0,\n", " 'attempts': [],\n", " 'duration': None,\n", " 'total_duration': 0,\n", " 'error': None,\n", " 'created_at': '2026-10-15T11:08:50.456001+00:00',\n", " 'completed_at': None},\n", " {'id': '6',\n", " 'status': 'pending',\n", " 'task': {'id': '6', 'value': 6},\n", " 'result': None,\n", " 'result_present': False,\n", " 'attempt_count': 0,\n", " 'failure_count': 0,\n", " 'attempts': [],\n", " 'duration': None,\n", " 'total_duration': 0,\n", " 'error': None,\n", " 'created_at': '2026-10-15T11:08:50.456079+00:00',\n", " 'completed_at': None},\n", " {'id': '7',\n", " 'status': 'pending',\n", " 'task': {'id': '7', 'value': 7},\n", " 'result': None,\n", " 'result_present': False,\n", " 'attempt_count': 0,\n", " 'failure_count': 0,\n", " 'attempts': [],\n", " 'duration': None,\n", " 'total_duration': 0,\n", " 'error': None,\n", " 'created_at': '2026-10-15T11:08:50.456155+00:00',\n", " 'completed_at': None},\n", " {'id': '8',\n", " 'status': 'pending',\n", " 'task': {'id': '8', 'value': 8},\n", " 'result': None,\n", " 'result_present': False,\n", " 'attempt_count': 0,\n", " 'failure_count': 0,\n", " 'attempts': [],\n", " 'duration': None,\n", " 'total_duration': 0,\n", " 'error': None,\n", " 'created_at': '2026-10-15T11:08:50.456228+00:00',\n", " 'completed_at': None},\n", " {'id': '9',\n", " 'status': 'pending',\n", " 'task': {'id': '9', 'value': 9},\n", " 'result': None,\n", " 'result_present': False,\n", " 'attempt_count': 0,\n", " 'failure_count': 0,\n", " 'attempts': [],\n", " 'duration': None,\n", " 'total_duration': 0,\n", " 'error': None,\n", " 'created_at': '2026-10-15T11:08:50.456301+00:00',\n", " 'completed_at': None},\n", " {'id': '10',\n", " 'status': 'pending',\n", " 'task': {'id': '10', 'value': 10},\n", " 'result': None,\n", " 'result_present': False,\n", " 'attempt_count': 0,\n", " 'failure_count': 0,\n", " 'attempts': [],\n", " 'duration': None,\n", " 'total_duration': 0,\n", " 'error': None,\n", " 'created_at': '2026-10-15T11:08:50.456375+00:00',\n", " 'completed_at': None}]" ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "results = %there get runner.results()\n", "results" ] }, { "cell_type": "markdown", "id": "877c9f93", "metadata": {}, "source": [ "To wait briefly before checking progress again:" ] }, { "cell_type": "code", "execution_count": 5, "id": "cde20679", "metadata": { "execution": { "iopub.execute_input": "2026-10-15T11:08:50.543558Z", "iopub.status.busy": "2026-10-15T11:08:50.543026Z", "iopub.status.idle": "2026-10-15T11:08:51.724071Z", "shell.execute_reply": "2026-10-15T11:08:51.722807Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "{'name': 'squares-example', 'state': 'running', 'total': 10, 'pending': 8, 'running': 1, 'completed': 1, 'failed': 0, 'progress': 0.1, 'current': '2', 'pause_requested': False, 'max_attempts': 3, 'exclusive': True, 'created_at': '2026-10-15T11:08:50.392116+00:00', 'updated_at': '2026-10-15T11:08:51.534568+00:00', 'metadata': {}, 'worker_error': None, 'path': '/data/user/0/me.herethere.pythonhere/files/task-runs/squares-example'}\n" ] } ], "source": [ "%%there --worker\n", "runner.wait(timeout=1)\n", "print(runner.status())" ] }, { "cell_type": "markdown", "id": "66103880", "metadata": {}, "source": [ "This waits at most one second and returns sooner if the worker has stopped.\n", "\n", "## Pause and resume (optional)\n", "\n", "While tasks are still running, you can pause them. `pause()` lets the current\n", "task finish, and `wait()` waits for the pause to take effect:" ] }, { "cell_type": "code", "execution_count": 6, "id": "1223fd66", "metadata": { "execution": { "iopub.execute_input": "2026-10-15T11:08:51.729600Z", "iopub.status.busy": "2026-10-15T11:08:51.729143Z", "iopub.status.idle": "2026-10-15T11:08:52.534555Z", "shell.execute_reply": "2026-10-15T11:08:52.534200Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "{'name': 'squares-example', 'state': 'paused', 'total': 10, 'pending': 8, 'running': 0, 'completed': 2, 'failed': 0, 'progress': 0.2, 'current': None, 'pause_requested': True, 'max_attempts': 3, 'exclusive': True, 'created_at': '2026-10-15T11:08:50.392116+00:00', 'updated_at': '2026-10-15T11:08:52.566010+00:00', 'metadata': {}, 'worker_error': None, 'path': '/data/user/0/me.herethere.pythonhere/files/task-runs/squares-example'}\n" ] } ], "source": [ "%%there --worker\n", "runner.pause()\n", "runner.wait()\n", "print(runner.status())" ] }, { "cell_type": "markdown", "id": "367abbc4", "metadata": {}, "source": [ "Resume any pending tasks with the same executor. Completed tasks are kept:" ] }, { "cell_type": "code", "execution_count": 7, "id": "b5eb02f7", "metadata": { "execution": { "iopub.execute_input": "2026-10-15T11:08:52.536295Z", "iopub.status.busy": "2026-10-15T11:08:52.536081Z", "iopub.status.idle": "2026-10-15T11:08:52.610231Z", "shell.execute_reply": "2026-10-15T11:08:52.609795Z" } }, "outputs": [], "source": [ "%%there\n", "runner.start(execute_task)" ] }, { "cell_type": "markdown", "id": "c1c3b7c5", "metadata": {}, "source": [ "If all tasks have already completed, there is nothing left to pause or resume.\n", "\n", "## Wait for completion\n", "\n", "Wait for all remaining work, then show the final counts. `--worker` keeps the\n", "app's UI responsive while this cell waits:" ] }, { "cell_type": "code", "execution_count": 8, "id": "bb575889", "metadata": { "execution": { "iopub.execute_input": "2026-10-15T11:08:52.612196Z", "iopub.status.busy": "2026-10-15T11:08:52.612021Z", "iopub.status.idle": "2026-10-15T11:09:03.087648Z", "shell.execute_reply": "2026-10-15T11:09:03.086969Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "State: completed\n", "Completed: 9 of 10\n", "Failed: 1\n" ] } ], "source": [ "%%there --worker\n", "runner.wait()\n", "status = runner.status()\n", "print(\"State:\", status[\"state\"])\n", "print(\"Completed:\", status[\"completed\"], \"of\", status[\"total\"])\n", "print(\"Failed:\", status[\"failed\"])" ] }, { "cell_type": "markdown", "id": "1277c976", "metadata": {}, "source": [ "Expected output:\n", "\n", "```text\n", "State: completed\n", "Completed: 9 of 10\n", "Failed: 1\n", "```\n", "\n", "`wait()` returns when the worker stops. A `completed` state means all tasks\n", "have been processed, including any that exhausted their retries; it does not\n", "mean every task succeeded. Inspect the deliberately failed task:" ] }, { "cell_type": "code", "execution_count": 9, "id": "77d99bda", "metadata": { "execution": { "iopub.execute_input": "2026-10-15T11:09:03.089909Z", "iopub.status.busy": "2026-10-15T11:09:03.089680Z", "iopub.status.idle": "2026-10-15T11:09:03.166070Z", "shell.execute_reply": "2026-10-15T11:09:03.165330Z" } }, "outputs": [ { "data": { "text/plain": [ "[{'id': '5',\n", " 'status': 'failed',\n", " 'task': {'id': '5', 'value': 5},\n", " 'result': None,\n", " 'result_present': False,\n", " 'attempt_count': 3,\n", " 'failure_count': 3,\n", " 'attempts': [{'number': 1,\n", " 'started_at': '2026-10-15T11:08:54.808851+00:00',\n", " 'finished_at': '2026-10-15T11:08:55.818719+00:00',\n", " 'duration': 1.0025099229533225,\n", " 'error': {'type': 'ValueError',\n", " 'message': 'Intentional example failure for value 5',\n", " 'traceback': 'Traceback (most recent call last):\\n File \"/home/user/hostcwd/.buildozer/android/app/tasks_here.py\", line 943, in _run_loop\\n File \"\", line 10, in execute_task\\nValueError: Intentional example failure for value 5\\n'}},\n", " {'number': 2,\n", " 'started_at': '2026-10-15T11:09:01.065718+00:00',\n", " 'finished_at': '2026-10-15T11:09:02.087962+00:00',\n", " 'duration': 1.0053778459550813,\n", " 'error': {'type': 'ValueError',\n", " 'message': 'Intentional example failure for value 5',\n", " 'traceback': 'Traceback (most recent call last):\\n File \"/home/user/hostcwd/.buildozer/android/app/tasks_here.py\", line 943, in _run_loop\\n File \"\", line 10, in execute_task\\nValueError: Intentional example failure for value 5\\n'}},\n", " {'number': 3,\n", " 'started_at': '2026-10-15T11:09:02.118377+00:00',\n", " 'finished_at': '2026-10-15T11:09:03.130124+00:00',\n", " 'duration': 1.0041442309739068,\n", " 'error': {'type': 'ValueError',\n", " 'message': 'Intentional example failure for value 5',\n", " 'traceback': 'Traceback (most recent call last):\\n File \"/home/user/hostcwd/.buildozer/android/app/tasks_here.py\", line 943, in _run_loop\\n File \"\", line 10, in execute_task\\nValueError: Intentional example failure for value 5\\n'}}],\n", " 'duration': 1.0041442309739068,\n", " 'total_duration': 3.0120319998823106,\n", " 'error': {'type': 'ValueError',\n", " 'message': 'Intentional example failure for value 5',\n", " 'traceback': 'Traceback (most recent call last):\\n File \"/home/user/hostcwd/.buildozer/android/app/tasks_here.py\", line 943, in _run_loop\\n File \"\", line 10, in execute_task\\nValueError: Intentional example failure for value 5\\n'},\n", " 'created_at': '2026-10-15T11:08:50.456001+00:00',\n", " 'completed_at': None}]" ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" } ], "source": [ "failures = %there get runner.failed()\n", "failures" ] }, { "cell_type": "markdown", "id": "37481f82", "metadata": {}, "source": [ "The failed record has `id: \"5\"`, `attempt_count: 3`, `failure_count: 3`,\n", "and the intentional `ValueError`. The other nine tasks have their square results.\n", "\n", "## Export results\n", "\n", "For larger results, export a JSON snapshot on the device and download the file:" ] }, { "cell_type": "code", "execution_count": 10, "id": "5ad203ee", "metadata": { "execution": { "iopub.execute_input": "2026-10-15T11:09:03.169068Z", "iopub.status.busy": "2026-10-15T11:09:03.168858Z", "iopub.status.idle": "2026-10-15T11:09:03.261067Z", "shell.execute_reply": "2026-10-15T11:09:03.260461Z" } }, "outputs": [], "source": [ "%%there --worker\n", "runner.export_results(\"task-results.json\")" ] }, { "cell_type": "code", "execution_count": 11, "id": "78f19640", "metadata": { "execution": { "iopub.execute_input": "2026-10-15T11:09:03.263735Z", "iopub.status.busy": "2026-10-15T11:09:03.263374Z", "iopub.status.idle": "2026-10-15T11:09:03.379833Z", "shell.execute_reply": "2026-10-15T11:09:03.379161Z" } }, "outputs": [], "source": [ "%there download task-results.json ./task-results.json" ] }, { "cell_type": "markdown", "id": "bb9ec543", "metadata": {}, "source": [ "Load the downloaded file in the notebook:" ] }, { "cell_type": "code", "execution_count": 12, "id": "b973158e", "metadata": { "execution": { "iopub.execute_input": "2026-10-15T11:09:03.383395Z", "iopub.status.busy": "2026-10-15T11:09:03.383092Z", "iopub.status.idle": "2026-10-15T11:09:03.387322Z", "shell.execute_reply": "2026-10-15T11:09:03.386570Z" } }, "outputs": [], "source": [ "import json\n", "\n", "with open(\"task-results.json\", encoding=\"utf-8\") as f:\n", " results = json.load(f)" ] }, { "cell_type": "markdown", "id": "3889aefb", "metadata": {}, "source": [ "The export includes all task records, including pending and failed tasks, in\n", "submission order. Export after completion for final results, or during execution\n", "for a snapshot. Relative paths use the device's current working directory,\n", "which is PythonHere's upload directory by default. For large runs, exporting\n", "and downloading a file avoids transferring all results through `%there get`.\n", "\n", "## Reopen after PythonHere restarts\n", "\n", "The task list and results are persistent, but Python callables and other runtime\n", "objects are not. After restarting PythonHere, reconnect, recreate the executor,\n", "open the named run, and start it again. Completed tasks are kept; only\n", "unfinished work runs again. If all ten tasks already finished, the run stays\n", "completed.\n", "\n", "In this walkthrough, all tasks have already finished. Reopening shows that\n", "the results and the failed task survived the restart. The task that exhausted\n", "its retries stays failed; calling `start()` does not retry it again. To try recovery of unfinished work separately,\n", "restart the app while a fresh run is executing, then reconnect and run this cell:" ] }, { "cell_type": "code", "execution_count": 13, "id": "b872d62b", "metadata": { "execution": { "iopub.execute_input": "2026-10-15T11:09:03.390443Z", "iopub.status.busy": "2026-10-15T11:09:03.390067Z", "iopub.status.idle": "2026-10-15T11:09:03.467955Z", "shell.execute_reply": "2026-10-15T11:09:03.467234Z" } }, "outputs": [], "source": [ "%%there\n", "import time\n", "\n", "from tasks_here import TaskRunner\n", "\n", "\n", "def execute_task(task, context):\n", " time.sleep(1)\n", " if task[\"value\"] == 5:\n", " raise ValueError(\"Intentional example failure for value 5\")\n", " return {\n", " \"value\": task[\"value\"],\n", " \"square\": task[\"value\"] ** 2,\n", " }\n", "\n", "\n", "runner = TaskRunner.open(\"squares-example\")\n", "runner.start(execute_task)" ] }, { "cell_type": "markdown", "id": "31e5ac4f", "metadata": {}, "source": [ "Wait for any unfinished work, then inspect the saved results:" ] }, { "cell_type": "code", "execution_count": 14, "id": "1f499cb8", "metadata": { "execution": { "iopub.execute_input": "2026-10-15T11:09:03.470435Z", "iopub.status.busy": "2026-10-15T11:09:03.470168Z", "iopub.status.idle": "2026-10-15T11:09:03.556049Z", "shell.execute_reply": "2026-10-15T11:09:03.555424Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "{'name': 'squares-example', 'state': 'completed', 'total': 10, 'pending': 0, 'running': 0, 'completed': 9, 'failed': 1, 'progress': 1.0, 'current': None, 'pause_requested': False, 'max_attempts': 3, 'exclusive': True, 'created_at': '2026-10-15T11:08:50.392116+00:00', 'updated_at': '2026-10-15T11:09:03.563013+00:00', 'metadata': {}, 'worker_error': None, 'path': '/data/user/0/me.herethere.pythonhere/files/task-runs/squares-example'}\n" ] } ], "source": [ "%%there --worker\n", "runner.wait()\n", "print(runner.status())" ] }, { "cell_type": "code", "execution_count": 15, "id": "bbe02163", "metadata": { "execution": { "iopub.execute_input": "2026-10-15T11:09:03.558836Z", "iopub.status.busy": "2026-10-15T11:09:03.558602Z", "iopub.status.idle": "2026-10-15T11:09:03.634568Z", "shell.execute_reply": "2026-10-15T11:09:03.633717Z" }, "tags": [ "hide-output" ] }, "outputs": [ { "data": { "text/plain": [ "[{'id': '1',\n", " 'status': 'done',\n", " 'task': {'id': '1', 'value': 1},\n", " 'result': {'value': 1, 'square': 1},\n", " 'result_present': True,\n", " 'attempt_count': 1,\n", " 'failure_count': 0,\n", " 'attempts': [{'number': 1,\n", " 'started_at': '2026-10-15T11:08:50.495452+00:00',\n", " 'finished_at': '2026-10-15T11:08:51.512079+00:00',\n", " 'duration': 1.0042130000074394,\n", " 'error': None}],\n", " 'duration': 1.0042130000074394,\n", " 'total_duration': 1.0042130000074394,\n", " 'error': None,\n", " 'created_at': '2026-10-15T11:08:50.455575+00:00',\n", " 'completed_at': '2026-10-15T11:08:51.512215+00:00'},\n", " {'id': '2',\n", " 'status': 'done',\n", " 'task': {'id': '2', 'value': 2},\n", " 'result': {'value': 2, 'square': 4},\n", " 'result_present': True,\n", " 'attempt_count': 1,\n", " 'failure_count': 0,\n", " 'attempts': [{'number': 1,\n", " 'started_at': '2026-10-15T11:08:51.532225+00:00',\n", " 'finished_at': '2026-10-15T11:08:52.544140+00:00',\n", " 'duration': 1.0043134610168636,\n", " 'error': None}],\n", " 'duration': 1.0043134610168636,\n", " 'total_duration': 1.0043134610168636,\n", " 'error': None,\n", " 'created_at': '2026-10-15T11:08:50.455746+00:00',\n", " 'completed_at': '2026-10-15T11:08:52.544266+00:00'},\n", " {'id': '3',\n", " 'status': 'done',\n", " 'task': {'id': '3', 'value': 3},\n", " 'result': {'value': 3, 'square': 9},\n", " 'result_present': True,\n", " 'attempt_count': 1,\n", " 'failure_count': 0,\n", " 'attempts': [{'number': 1,\n", " 'started_at': '2026-10-15T11:08:52.716334+00:00',\n", " 'finished_at': '2026-10-15T11:08:53.739693+00:00',\n", " 'duration': 1.0050933079910465,\n", " 'error': None}],\n", " 'duration': 1.0050933079910465,\n", " 'total_duration': 1.0050933079910465,\n", " 'error': None,\n", " 'created_at': '2026-10-15T11:08:50.455842+00:00',\n", " 'completed_at': '2026-10-15T11:08:53.739822+00:00'},\n", " {'id': '4',\n", " 'status': 'done',\n", " 'task': {'id': '4', 'value': 4},\n", " 'result': {'value': 4, 'square': 16},\n", " 'result_present': True,\n", " 'attempt_count': 1,\n", " 'failure_count': 0,\n", " 'attempts': [{'number': 1,\n", " 'started_at': '2026-10-15T11:08:53.768668+00:00',\n", " 'finished_at': '2026-10-15T11:08:54.782363+00:00',\n", " 'duration': 1.0052369229961187,\n", " 'error': None}],\n", " 'duration': 1.0052369229961187,\n", " 'total_duration': 1.0052369229961187,\n", " 'error': None,\n", " 'created_at': '2026-10-15T11:08:50.455924+00:00',\n", " 'completed_at': '2026-10-15T11:08:54.782494+00:00'},\n", " {'id': '5',\n", " 'status': 'failed',\n", " 'task': {'id': '5', 'value': 5},\n", " 'result': None,\n", " 'result_present': False,\n", " 'attempt_count': 3,\n", " 'failure_count': 3,\n", " 'attempts': [{'number': 1,\n", " 'started_at': '2026-10-15T11:08:54.808851+00:00',\n", " 'finished_at': '2026-10-15T11:08:55.818719+00:00',\n", " 'duration': 1.0025099229533225,\n", " 'error': {'type': 'ValueError',\n", " 'message': 'Intentional example failure for value 5',\n", " 'traceback': 'Traceback (most recent call last):\\n File \"/home/user/hostcwd/.buildozer/android/app/tasks_here.py\", line 943, in _run_loop\\n File \"\", line 10, in execute_task\\nValueError: Intentional example failure for value 5\\n'}},\n", " {'number': 2,\n", " 'started_at': '2026-10-15T11:09:01.065718+00:00',\n", " 'finished_at': '2026-10-15T11:09:02.087962+00:00',\n", " 'duration': 1.0053778459550813,\n", " 'error': {'type': 'ValueError',\n", " 'message': 'Intentional example failure for value 5',\n", " 'traceback': 'Traceback (most recent call last):\\n File \"/home/user/hostcwd/.buildozer/android/app/tasks_here.py\", line 943, in _run_loop\\n File \"\", line 10, in execute_task\\nValueError: Intentional example failure for value 5\\n'}},\n", " {'number': 3,\n", " 'started_at': '2026-10-15T11:09:02.118377+00:00',\n", " 'finished_at': '2026-10-15T11:09:03.130124+00:00',\n", " 'duration': 1.0041442309739068,\n", " 'error': {'type': 'ValueError',\n", " 'message': 'Intentional example failure for value 5',\n", " 'traceback': 'Traceback (most recent call last):\\n File \"/home/user/hostcwd/.buildozer/android/app/tasks_here.py\", line 943, in _run_loop\\n File \"\", line 10, in execute_task\\nValueError: Intentional example failure for value 5\\n'}}],\n", " 'duration': 1.0041442309739068,\n", " 'total_duration': 3.0120319998823106,\n", " 'error': {'type': 'ValueError',\n", " 'message': 'Intentional example failure for value 5',\n", " 'traceback': 'Traceback (most recent call last):\\n File \"/home/user/hostcwd/.buildozer/android/app/tasks_here.py\", line 943, in _run_loop\\n File \"\", line 10, in execute_task\\nValueError: Intentional example failure for value 5\\n'},\n", " 'created_at': '2026-10-15T11:08:50.456001+00:00',\n", " 'completed_at': None},\n", " {'id': '6',\n", " 'status': 'done',\n", " 'task': {'id': '6', 'value': 6},\n", " 'result': {'value': 6, 'square': 36},\n", " 'result_present': True,\n", " 'attempt_count': 1,\n", " 'failure_count': 0,\n", " 'attempts': [{'number': 1,\n", " 'started_at': '2026-10-15T11:08:55.851173+00:00',\n", " 'finished_at': '2026-10-15T11:08:56.865618+00:00',\n", " 'duration': 1.0033425380242988,\n", " 'error': None}],\n", " 'duration': 1.0033425380242988,\n", " 'total_duration': 1.0033425380242988,\n", " 'error': None,\n", " 'created_at': '2026-10-15T11:08:50.456079+00:00',\n", " 'completed_at': '2026-10-15T11:08:56.865746+00:00'},\n", " {'id': '7',\n", " 'status': 'done',\n", " 'task': {'id': '7', 'value': 7},\n", " 'result': {'value': 7, 'square': 49},\n", " 'result_present': True,\n", " 'attempt_count': 1,\n", " 'failure_count': 0,\n", " 'attempts': [{'number': 1,\n", " 'started_at': '2026-10-15T11:08:56.889569+00:00',\n", " 'finished_at': '2026-10-15T11:08:57.903773+00:00',\n", " 'duration': 1.0033163069747388,\n", " 'error': None}],\n", " 'duration': 1.0033163069747388,\n", " 'total_duration': 1.0033163069747388,\n", " 'error': None,\n", " 'created_at': '2026-10-15T11:08:50.456155+00:00',\n", " 'completed_at': '2026-10-15T11:08:57.903901+00:00'},\n", " {'id': '8',\n", " 'status': 'done',\n", " 'task': {'id': '8', 'value': 8},\n", " 'result': {'value': 8, 'square': 64},\n", " 'result_present': True,\n", " 'attempt_count': 1,\n", " 'failure_count': 0,\n", " 'attempts': [{'number': 1,\n", " 'started_at': '2026-10-15T11:08:57.933493+00:00',\n", " 'finished_at': '2026-10-15T11:08:58.948942+00:00',\n", " 'duration': 1.0050470000132918,\n", " 'error': None}],\n", " 'duration': 1.0050470000132918,\n", " 'total_duration': 1.0050470000132918,\n", " 'error': None,\n", " 'created_at': '2026-10-15T11:08:50.456228+00:00',\n", " 'completed_at': '2026-10-15T11:08:58.949102+00:00'},\n", " {'id': '9',\n", " 'status': 'done',\n", " 'task': {'id': '9', 'value': 9},\n", " 'result': {'value': 9, 'square': 81},\n", " 'result_present': True,\n", " 'attempt_count': 1,\n", " 'failure_count': 0,\n", " 'attempts': [{'number': 1,\n", " 'started_at': '2026-10-15T11:08:58.979740+00:00',\n", " 'finished_at': '2026-10-15T11:08:59.994785+00:00',\n", " 'duration': 1.0035531540052034,\n", " 'error': None}],\n", " 'duration': 1.0035531540052034,\n", " 'total_duration': 1.0035531540052034,\n", " 'error': None,\n", " 'created_at': '2026-10-15T11:08:50.456301+00:00',\n", " 'completed_at': '2026-10-15T11:08:59.994928+00:00'},\n", " {'id': '10',\n", " 'status': 'done',\n", " 'task': {'id': '10', 'value': 10},\n", " 'result': {'value': 10, 'square': 100},\n", " 'result_present': True,\n", " 'attempt_count': 1,\n", " 'failure_count': 0,\n", " 'attempts': [{'number': 1,\n", " 'started_at': '2026-10-15T11:09:00.027395+00:00',\n", " 'finished_at': '2026-10-15T11:09:01.040481+00:00',\n", " 'duration': 1.0060150770004839,\n", " 'error': None}],\n", " 'duration': 1.0060150770004839,\n", " 'total_duration': 1.0060150770004839,\n", " 'error': None,\n", " 'created_at': '2026-10-15T11:08:50.456375+00:00',\n", " 'completed_at': '2026-10-15T11:09:01.040608+00:00'}]" ] }, "execution_count": 15, "metadata": {}, "output_type": "execute_result" } ], "source": [ "saved_results = %there get runner.results()\n", "saved_results" ] }, { "cell_type": "markdown", "id": "b68d0b26", "metadata": {}, "source": [ "Execution is at least once. If PythonHere stops after an executor begins but\n", "before its result is committed, that task can run again. Executors with\n", "irreversible side effects should therefore be idempotent or check durable state\n", "before applying those effects.\n", "\n", "## Optional: reuse runtime resources\n", "\n", "The executor always receives a second argument. It is `None` when `start()` has\n", "no setup callback. For expensive runtime resources, create them once per worker\n", "session and reuse the returned context for every task and retry:\n", "\n", "```python\n", "def setup():\n", " return create_runtime()\n", "\n", "\n", "def execute_with_runtime(task, runtime):\n", " return runtime.process(task)\n", "\n", "\n", "runner.start(execute_with_runtime, setup=setup)\n", "```\n", "\n", "Setup runs in the background worker thread, so `start()` still returns promptly.\n", "While setup executes, `runner.status()[\"state\"]` is `\"setting_up\"`; after setup\n", "succeeds it becomes `\"running\"` before the first task is claimed.\n", "If pause is requested while setup is running, setup finishes and the worker exits\n", "before claiming a task. A later `start()` calls setup again. Context is runtime\n", "state: it is not persisted or included in results, and the runner does not close\n", "it automatically. Applications that require deterministic resource cleanup must\n", "manage that cleanup themselves.\n", "\n", "## Clean up\n", "\n", "When you are done inspecting the completed run, you can delete it to repeat\n", "this example with the same name. Skip this cell to keep its saved results.\n", "This deletes only `\"squares-example\"` and its tasks/results; other runs and\n", "exported JSON files are untouched." ] }, { "cell_type": "code", "execution_count": 16, "id": "7084f40a", "metadata": { "execution": { "iopub.execute_input": "2026-10-15T11:09:03.638190Z", "iopub.status.busy": "2026-10-15T11:09:03.637937Z", "iopub.status.idle": "2026-10-15T11:09:03.739662Z", "shell.execute_reply": "2026-10-15T11:09:03.738898Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Example run deleted. You can now run the example again.\n" ] } ], "source": [ "%%there --worker\n", "from tasks_here import TaskRunner, TaskRunNotFoundError\n", "\n", "try:\n", " saved_run = TaskRunner.open(\"squares-example\")\n", "except TaskRunNotFoundError:\n", " print(\"The example run has already been deleted.\")\n", "else:\n", " saved_run.delete()\n", " print(\"Example run deleted. You can now run the example again.\")" ] }, { "cell_type": "markdown", "id": "aa6cbfcd", "metadata": {}, "source": [ "A running task run cannot be deleted. Finish or pause it first." ] } ], "metadata": { "jupytext": { "default_lexer": "python" }, "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "there-ipython3", "version": "3.12.3" } }, "nbformat": 4, "nbformat_minor": 5 }