Client¶
tdworkflow.client module¶
-
class
tdworkflow.client.
AttemptAPI
¶ Bases:
object
-
attempt
(attempt: Union[int, tdworkflow.attempt.Attempt], inplace: bool = False) → Optional[tdworkflow.attempt.Attempt]¶ Get an attempt
Parameters: - attempt (int) – Attempt ID or Attempt object
- inplace – If True, do operation inplace and return None
Returns: Attempt object
Return type: Attempt
-
attempt_tasks
(attempt: Union[int, tdworkflow.attempt.Attempt]) → List[tdworkflow.task.Task]¶ Get tasks of a session
Parameters: attempt – Attempt id or Attempt object Returns: List of Task
-
attempts
(project: Union[str, tdworkflow.project.Project, None] = None, workflow: Union[str, tdworkflow.workflow.Workflow, None] = None, include_retried: Optional[bool] = None, last_id: Optional[int] = None, page_size: Optional[int] = None) → List[tdworkflow.attempt.Attempt]¶ List attempts
Parameters: - project (Optional[Union[str, Project]]) – Project name or Project object, optional
- workflow (Optional[Union[str, Workflow]]) – Workflow name or Workflow object, optional
- include_retried (Optional[bool]) – Flag to include retried
- last_id (Optional[int]) – Last ID
- page_size (Optional[int]) – Page size
Returns: List of Attempt object
Return type: List[Attempt]
-
kill_attempt
(attempt: Union[int, tdworkflow.attempt.Attempt], inplace: bool = False) → Optional[tdworkflow.attempt.Attempt]¶ Kill a session
Parameters: - attempt (Union[int, Attempt]) – Attempt ID or Attempt object
- inplace – If True, do operation inplace and return None
Returns: True
if succeededReturn type:
-
retried_attempts
(attempt: Union[int, tdworkflow.attempt.Attempt]) → List[tdworkflow.attempt.Attempt]¶ Get retried attempt list
Parameters: attempt – Attempt id or Attempt object Returns: List of Attempt
-
start_attempt
(workflow: Union[int, tdworkflow.workflow.Workflow], session_time: Optional[str] = None, retry_attempt_name: Optional[str] = None, workflow_params: Optional[Dict[str, Any]] = None) → tdworkflow.attempt.Attempt¶ Start workflow session
Parameters: - workflow – Workflow id or Workflow object
- session_time – Session time, optional Default:
datetime.datetime.now()
- retry_attempt_name – Retry attempt name, optional
- workflow_params – Extra workflow parameters
Returns:
-
wait_attempt
(attempt: Union[int, tdworkflow.attempt.Attempt], wait_interval: int = 5) → tdworkflow.attempt.Attempt¶ Wait until an attempt finished
Parameters: - attempt (Union[int, Attempt]) – Attempt ID or Attempt object
- wait_interval (int) – Wait interval in second. Default 5 sec
Returns: Latest status of Attempt
Return type:
-
-
class
tdworkflow.client.
Client
(site: str = 'us', endpoint: Optional[str] = None, apikey: Optional[str] = None, user_agent: Optional[str] = None, _session: Optional[requests.sessions.Session] = None, scheme: str = 'https')¶ Bases:
tdworkflow.client.AttemptAPI
,tdworkflow.client.WorkflowAPI
,tdworkflow.client.ProjectAPI
,tdworkflow.client.ScheduleAPI
,tdworkflow.client.SessionAPI
,tdworkflow.client.LogAPI
-
delete
(path: str, params: Optional[Dict[str, str]] = None) → Optional[Dict[str, str]]¶ DELETE operator for REST API
Parameters: - path (str) – Treasure Workflow API path
- params (Optional[Dict[str, str]], optional) – Query parameters, defaults to None
Returns: True
if succeededReturn type: bool
-
get
(path: str, params: Optional[Dict[str, str]] = None, content: bool = False) → Union[Dict[str, str], bytes]¶ GET operator for REST API
Parameters: - path (str) – Treasure Workflow API path
- params (Optional[Dict[str, str]], optional) – Query parameters, defaults to None
- content (bool) – Return content body without parsing JSON if
True
Returns: Response data in JSON or bytes
Return type: Union[Dict[str, str], bytes]
-
http
¶ Returns: Established session Return type: requests.Session
-
post
(path: str, body: Optional[Dict[str, str]] = None, content: bool = False) → Union[Dict[str, str], bytes, None]¶ POST operator for REST API
Parameters: - path (str) – Treasure Workflow API path
- body (Optional[Dict[str, str]], optional) – Content body in dictionary to be passed in JSON
- content (bool) – Return content body without parsing JSON if
True
Returns: True
if succeeded
-
put
(path: str, data: Union[Dict[KT, VT], List[Tuple], BinaryIO, None] = None, _json: Optional[Dict[str, str]] = None, params: Optional[Dict[str, str]] = None) → Optional[Dict[str, str]]¶ PUT operator for REST API
Parameters: - path (str) – Treasure Workflow API path
- data (Optional[Union[Dict, List[Tuple], BinaryIO]], oprional) – Content body
- _json (Optional[Dict[str, str]], optional) – Content body as JSON
- params (Optional[Dict[str, str]], optional) – Query parameters
Returns: Response content
Return type: Dict[str,str]
-
-
class
tdworkflow.client.
LogAPI
¶ Bases:
object
-
log_file
(attempt: Union[tdworkflow.attempt.Attempt, int], file: Union[tdworkflow.log.LogFile, str]) → str¶ Get a log string for an attempt
Parameters: - attempt – Target Attempt id or Attempt object
- file – LogFile name or LogFile object
Returns: Log string
-
log_files
(attempt: Union[tdworkflow.attempt.Attempt, int], task: Optional[str] = None, direct_download: Optional[bool] = None) → List[tdworkflow.log.LogFile]¶ Get log files information
Parameters: - attempt – Target Attempt id or Attempt object
- task – Task name
- direct_download – Flag for direct download
Returns: List of LogFile
-
logs
(attempt: Union[tdworkflow.attempt.Attempt, int]) → List[str]¶ Get log string list for an attempt
Parameters: attempt – Attempt ID or Attempt object Returns: A list of log >>> import tdworkflow >>> client = tdworkflow.client.Client("us") >>> attempts = client.attempts(project="pandas-df") >>> logs = client.logs(attempts[0]) >>> print(logs) ['2019-10-30 08:34:51.672 +0000 [INFO] (0250@[1:pandas-df]+pandas-df+read_into_df) io.digdag.core.agent.OperatorManager: py>: py_scripts.examples.read_td_table\n', '2019-10-30 08:34:59.879 +0000 [INFO] (0237@[1:pandas-df]+pandas-df+read_into_df) io.digdag.core.agent.OperatorManager: py>: py_scripts.examples.read_td_table\nWait running a command task: status provisioning', ...
-
-
class
tdworkflow.client.
ProjectAPI
¶ Bases:
object
-
create_project
(project_name: str, target_dir: str, exclude_patterns: Optional[List[str]] = None, revision: Optional[str] = None) → tdworkflow.project.Project¶ Create a new project
Parameters: - project_name – Project name
- target_dir – Target directory name
- exclude_patterns – Exclude file patterns. They are treated as regexp patterns. default: [“venv”, “.venv”, “__pycache__”, “.egg-info”, “.digdag”, “.pyc”] + dot files
- revision – Revision name
Returns:
-
delete_project
(project: Union[int, tdworkflow.project.Project]) → bool¶ Delete a project
Parameters: project – Project id or Project object Returns: True
if succeeded
-
delete_secret
(project: Union[int, tdworkflow.project.Project], key: str) → bool¶ Delete secret key
Parameters: - project (Union[int, Project]) – Project ID or Project object
- key (str) – Secret key to be deleted
Returns: True
if succeededReturn type: bool
-
delete_secrets
(project: Union[int, tdworkflow.project.Project], keys: List[str]) → bool¶ Delete multiple secret keys at once
Parameters: - project (Union[int, Project]) – Project ID or Project object
- keys (List[str]) – The list of secret keys to be deleted
Returns: True
if succeededReturn type: bool
-
download_project_archive
(project: Union[int, tdworkflow.project.Project], file_path: str, revision: Optional[str] = None) → bool¶ Download a project and save as a file (tar.gz)
Parameters: - project – Project id or Project object
- file_path – Target file path to be saved in tar.gz
- revision – Revision name
Returns: True
if succeeded
-
project
(project: Union[int, tdworkflow.project.Project]) → tdworkflow.project.Project¶ Get a project
Parameters: project (Union[int, Project]) – Project id or Project object Returns: A Project
-
project_revisions
(project: Union[int, tdworkflow.project.Project]) → List[tdworkflow.revision.Revision]¶ List revisions associated with Project
Parameters: project – Project id or Project object Returns: List of Revision
-
project_schedules
(project: Union[int, tdworkflow.project.Project], workflow: Union[str, tdworkflow.workflow.Workflow, None] = None, last_id: Optional[int] = None) → List[tdworkflow.schedule.Schedule]¶ List schedules associated with Project
Parameters: - project – Project ID or project object
- workflow – Workflow name or Workflow object
- last_id – Last ID
Returns: List of Schedule
-
project_sessions
(project: Union[int, tdworkflow.project.Project], workflow: Union[str, tdworkflow.workflow.Workflow, None] = None, last_id: Optional[int] = None, page_size: Optional[int] = None) → List[tdworkflow.session.Session]¶ List sessions associated with a Project
Parameters: - project – Project ID or Project object
- workflow – Workflow name or Workflow object
- last_id – Last ID
- page_size – Page size
Returns: List of Session
-
project_workflows
(project: Union[int, tdworkflow.project.Project], workflow: Union[str, tdworkflow.workflow.Workflow, None] = None, revision: Optional[str] = None) → List[tdworkflow.workflow.Workflow]¶ Get workflows associated with a project
Parameters: - project (Optional[Union[str, Workflow]], optional) – Project id or Project object
- workflow – Workflow name or Workflow object
- revision (Optional[str], optional) – Revision name
Returns: List of Workflow
Return type: List[Workflow]
-
project_workflows_by_name
(project_name: str) → List[tdworkflow.workflow.Workflow]¶ List workflows associate with Project by project name
Parameters: project_name – Target project name Returns: List of Workflow
-
projects
(name: Optional[str] = None) → List[tdworkflow.project.Project]¶ List projects
Parameters: name (Optional[str], optional) – Project name Returns: List of Project Return type: List[Project]
-
-
class
tdworkflow.client.
ScheduleAPI
¶ Bases:
object
-
backfill_schedule
(schedule: Union[int, tdworkflow.schedule.Schedule], attempt_name: str, from_time: Union[str, datetime.datetime], dry_run: bool = False, count: Optional[int] = None) → tdworkflow.schedule.ScheduleAttempt¶ Run or re-run past schedules
Parameters: - schedule – Target Schedule id or Schedule object
- attempt_name – Attempt name
- from_time – From time e.g “2019-11-01T06:20:07.000+00:00” in
str
ordatetime.datetime
. - dry_run – Flag for dry run
- count – Count
Returns: ScheduleAttempt
-
disable_schedule
(schedule: Union[int, tdworkflow.schedule.Schedule]) → tdworkflow.schedule.Schedule¶ Disable a schedule
Parameters: schedule – Schedule ID or Schedule object Returns: New Schedule
-
enable_schedule
(schedule: Union[int, tdworkflow.schedule.Schedule]) → tdworkflow.schedule.Schedule¶ Enable a schedule
Parameters: schedule – Schedule ID or Schedule object Returns: New Schedule
-
schedule
(schedule: Union[int, tdworkflow.schedule.Schedule]) → tdworkflow.schedule.Schedule¶ Get a schedule
Parameters: schedule – Schedule id or Schedule object Returns: Schedule
-
schedules
(last_id: Optional[int] = None) → List[tdworkflow.schedule.Schedule]¶ List schedules
Parameters: last_id – Last ID Returns: List of Schedule
-
skip_schedule
(schedule: Union[int, tdworkflow.schedule.Schedule], from_time: Union[str, datetime.datetime, None] = None, next_time: Optional[str] = None, next_run_time: Union[str, datetime.datetime, None] = None, dry_run: Optional[bool] = False) → tdworkflow.schedule.Schedule¶ Skip schedules forward to a future time
Parameters: - schedule – Schedule ID or Schedule object
- from_time – From time
- next_time – Next time
- next_run_time – Next run time
- dry_run – Flag for dry run
Returns: New Schedule
-
-
class
tdworkflow.client.
SessionAPI
¶ Bases:
object
-
session
(session: Union[int, tdworkflow.session.Session]) → tdworkflow.session.Session¶ Get a session
Parameters: session – Sesion ID or Session object Returns: New Session
-
session_attempts
(session: Union[int, tdworkflow.session.Session], last_id: Optional[int] = None, page_size: Optional[int] = None) → List[tdworkflow.attempt.Attempt]¶ Get attempts of a session
Parameters: - session – Session ID or Session object
- last_id – Last ID
- page_size – Page size
Returns: List of Attempt
-
sessions
(last_id: Optional[int] = None, page_size: Optional[int] = None) → List[tdworkflow.session.Session]¶ List sessions
Parameters: - last_id – Last ID
- page_size – Page size
Returns: List of Session
-