Spec of user job. More...

#include <operation.h>

Public Member Functions
TSelf &	AddLocalFile (const TLocalFilePath &path, const TAddLocalFileOptions &options=TAddLocalFileOptions())
	Specify a local file to upload to Cypress and prepare for use in job.
TVector< std::tuple< TLocalFilePath, TAddLocalFileOptions > >	GetLocalFiles () const
	Get the list of all added local files.
	FLUENT_VECTOR_FIELD (TRichYPath, File)
	Paths to files in Cypress to use in job.
	FLUENT_VECTOR_FIELD (TYPath, Layer)
	Porto layers to use in the job. Layers are listed from top to bottom.
TSelf &	DockerImage (TString)
	Docker image to use in the job.
TSelf &	MemoryLimit (i64)
	MemoryLimit specifies how much memory job process can use.
TSelf &	ExtraTmpfsSize (i64)
	Size of data that is going to be written to tmpfs.
TSelf &	CpuLimit (double)
	Maximum number of CPU cores for a single job to use.
TSelf &	MemoryReserveFactor (double)
	Fraction of NYT::TUserJobSpec::MemoryLimit that job gets at start.
TUserJobSpec &	JobBinaryLocalPath (TString path, TMaybe< TString > md5=Nothing())
	Local path to executable to be used inside jobs.
TUserJobSpec &	JobBinaryCypressPath (TString path, TMaybe< TTransactionId > transactionId=Nothing())
	Cypress path to executable to be used inside jobs.
TSelf &	JobCommandPrefix (TString)
	String that will be prepended to the command.
TSelf &	JobCommandSuffix (TString)
	String that will be appended to the command.
	FLUENT_MAP_FIELD (TString, TString, Environment)
	Map of environment variables that will be set for jobs.
TSelf &	DiskSpaceLimit (ui64)
	Limit for all files inside job sandbox (in bytes).
TSelf &	PortCount (ui16)
	Number of ports reserved for the job (passed through environment in YT_PORT_0, YT_PORT_1, ...).
TSelf &	NetworkProject (TString)
	Network project used to isolate job network.
TSelf &	JobTimeLimit (TDuration)
	Limit on job execution time.
const TJobBinaryConfig &	GetJobBinary () const
	Get job binary config.
	FLUENT_VECTOR_FIELD (TJobProfilerSpec, JobProfiler)
	List of profilers to run.
TSelf &	DiskRequest (TDiskRequest)
	Specification of a disk required for job.
TSelf &	EnableRpcProxyInJobProxy (bool)
	Activates the RPC proxy within the job proxy.

Detailed Description

Spec of user job.

See also: https://ytsaurus.tech/docs/en/user-guide/data-processing/operations/operations-options#user_script_options

Member Function Documentation

◆ DiskRequest()

TSelf & NYT::TUserJobSpec::DiskRequest ( TDiskRequest )

Specification of a disk required for job.

See also: https://ytsaurus.tech/docs/en/user-guide/data-processing/operations/operations-options#disk_request

◆ EnableRpcProxyInJobProxy()

TSelf & NYT::TUserJobSpec::EnableRpcProxyInJobProxy ( bool )

Activates the RPC proxy within the job proxy.

By enabling this option, the environment variable YT_JOB_PROXY_SOCKET_PATH will be set. You can use this variable to obtain the unix domain socket path and then construct a client for sending requests.

Note: Do not enable this option without prior discussion with the YTsaurus team.

◆ ExtraTmpfsSize()

TSelf & NYT::TUserJobSpec::ExtraTmpfsSize ( i64 )

Size of data that is going to be written to tmpfs.

This option should be used if job writes data to tmpfs.

ExtraTmpfsSize should not include size of files specified with NYT::TUserJobSpec::AddLocalFile or NYT::TUserJobSpec::AddFile These files are copied to tmpfs automatically and their total size is computed automatically.

See also: NYT::TOperationOptions::MountSandboxInTmpfs; NYT::TUserJobSpec::MemoryLimit

◆ JobBinaryLocalPath()

TUserJobSpec & NYT::TUserJobSpec::JobBinaryLocalPath	(	TString	path,
		TMaybe< TString >	md5 = Nothing() )

Local path to executable to be used inside jobs.

Provided executable must use C++ YT API library (this library) and implement job class that is going to be used.

This option might be useful if we want to start operation from nonlinux machines (in that case we use JobBinary to provide path to the same program compiled for linux). Other example of using this option is uploading executable to cypress in advance and save the time required to upload current executable to cache. md5 argument can be used to save cpu time and disk IO when binary MD5 checksum is known. When argument is not provided library will compute it itself.

◆ JobCommandPrefix()

TSelf & NYT::TUserJobSpec::JobCommandPrefix ( TString )

String that will be prepended to the command.

This option overrides NYT::TOperationOptions::JobCommandPrefix.

◆ JobCommandSuffix()

TSelf & NYT::TUserJobSpec::JobCommandSuffix ( TString )

String that will be appended to the command.

This option overrides NYT::TOperationOptions::JobCommandSuffix.

◆ JobTimeLimit()

TSelf & NYT::TUserJobSpec::JobTimeLimit ( TDuration )

Limit on job execution time.

Jobs that exceed this limit will be considered failed.

◆ MemoryLimit()

TSelf & NYT::TUserJobSpec::MemoryLimit ( i64 )

MemoryLimit specifies how much memory job process can use.

Note

If job uses tmpfs (check NYT::TOperationOptions::MountSandboxInTmpfs) YT computes its memory usage as total of:

memory usage of job process itself (including mapped files);
total size of tmpfs used by this job.

When NYT::TOperationOptions::MountSandboxInTmpfs is enabled library will compute total size of all files used by this job and add this total size to MemoryLimit. Thus, you shouldn't include size of your files (e.g. binary file) into MemoryLimit.

Final memory memory_limit passed to YT is calculated as follows:

memory_limit = MemoryLimit + <total-size-of-used-files> + ExtraTmpfsSize

NYT::TUserJobSpec::ExtraTmpfsSize

TSelf & ExtraTmpfsSize(i64)

Size of data that is going to be written to tmpfs.

NYT::TUserJobSpec::MemoryLimit

TSelf & MemoryLimit(i64)