Options for NYT::IOperationClient::Map and other operation start commands. More...

#include <operation.h>

Public Types
enum class	EStartOperationMode : int { AsyncPrepare , AsyncStart , SyncStart , SyncWait }
	Start operation mode. More...
enum class	EFileCacheMode : int { ApiCommandBased , CachelessRandomPathUpload }
	File cache mode. More...

Public Member Functions
TSelf &	Spec (TNode)
	Additional field to put to operation spec.
TSelf &	StartOperationMode (EStartOperationMode)
	Start operation mode.
TSelf &	Wait (bool value)
	Wait for operation finish synchronously.
TSelf &	UseTableFormats (bool)
	Use format from table attribute (for YAMR-like format).
TSelf &	JobCommandPrefix (TString)
	Prefix for bash command running the jobs.
TSelf &	JobCommandSuffix (TString)
	Suffix for bash command running the jobs.
TSelf &	MountSandboxInTmpfs (bool)
	Put all files required by the job into tmpfs.
TSelf &	FileStorage (TString)
	Path to directory to store temporary files. Useful if you want to control how lifetime of uploaded files.
TSelf &	FileExpirationTimeout (TDuration)
	Expiration timeout for uploaded files.
TSelf &	SecureVault (TNode)
	Info to be passed securely to the job.
TSelf &	FileCacheMode (EFileCacheMode)
	File cache mode.
TSelf &	FileStorageTransactionId (TTransactionId)
	Id of transaction within which all Cypress file storage entries will be checked/created.
TSelf &	CreateDebugOutputTables (bool)
	Ensure stderr and core tables exist before starting operation.
TSelf &	CreateOutputTables (bool)
	Ensure output tables exist before starting operation.
TSelf &	InferOutputSchema (bool)
	Try to infer schema of inexistent table from the type of written rows.
TSelf &	MinJobStateSizeToPassViaFile (i64)
	If job state size is less than specified value, job state will be passed via environment variable in spec.

Detailed Description

Options for NYT::IOperationClient::Map and other operation start commands.

Member Enumeration Documentation

◆ EFileCacheMode

enum class NYT::TOperationOptions::EFileCacheMode : int

strong

File cache mode.

Enumerator
ApiCommandBased	Use YT API commands "get_file_from_cache" and "put_file_to_cache".
CachelessRandomPathUpload	Upload files to random paths inside NYT::TOperationOptions::FileStorage without caching.

◆ EStartOperationMode

enum class NYT::TOperationOptions::EStartOperationMode : int

strong

Start operation mode.

Enumerator
AsyncPrepare	Prepare operation asynchronously. Call IOperation::Start() to start operation.
AsyncStart	Prepare and start operation asynchronously. Don't wait for operation completion.
SyncStart	Prepare and start operation synchronously. Don't wait for operation completion.
SyncWait	Prepare, start and wait for operation completion synchronously.

Member Function Documentation

◆ CreateDebugOutputTables()

TSelf & NYT::TOperationOptions::CreateDebugOutputTables ( bool )

Ensure stderr and core tables exist before starting operation.

If set to false, it is user's responsibility to ensure these tables exist.

◆ CreateOutputTables()

TSelf & NYT::TOperationOptions::CreateOutputTables ( bool )

Ensure output tables exist before starting operation.

If set to false, it is user's responsibility to ensure output tables exist.

◆ FileExpirationTimeout()

TSelf & NYT::TOperationOptions::FileExpirationTimeout ( TDuration )

Expiration timeout for uploaded files.

Set attribute ExpirationTimeout for files being uploaded during operation preparation. Useful when using custom FileStorage and don't want to create separate cleanup process.

When using default FileStorage inside //tmp this parameter is almost useless. //tmp directory is cleaned up by separate process and files can be deleted before FileExpiratoinTimeout is reached.

◆ FileStorageTransactionId()

TSelf & NYT::TOperationOptions::FileStorageTransactionId ( TTransactionId )

Id of transaction within which all Cypress file storage entries will be checked/created.

By default, the root transaction is used.

Note

Set a specific transaction only if you

specify non-default file storage path in NYT::TOperationOptions::FileStorage or in NYT::TConfig::RemoteTempFilesDirectory.
use CachelessRandomPathUpload caching mode (NYT::TOperationOptions::FileCacheMode).

◆ InferOutputSchema()

TSelf & NYT::TOperationOptions::InferOutputSchema ( bool )

Try to infer schema of inexistent table from the type of written rows.

Note: Default values for this option may differ depending on the row type. For protobuf it's currently false by default.

◆ JobCommandPrefix()

TSelf & NYT::TOperationOptions::JobCommandPrefix ( TString )

Prefix for bash command running the jobs.

Can be overridden for the specific job type in the NYT::TUserJobSpec.

◆ JobCommandSuffix()

TSelf & NYT::TOperationOptions::JobCommandSuffix ( TString )

Suffix for bash command running the jobs.

Can be overridden for the specific job type in the NYT::TUserJobSpec.

◆ MinJobStateSizeToPassViaFile()

TSelf & NYT::TOperationOptions::MinJobStateSizeToPassViaFile ( i64 )

If job state size is less than specified value, job state will be passed via environment variable in spec.

Note: Default value is 0, so job spec is passed via file

◆ MountSandboxInTmpfs()

TSelf & NYT::TOperationOptions::MountSandboxInTmpfs ( bool )

Put all files required by the job into tmpfs.

This option can be set globally using NYT::TConfig::MountSandboxInTmpfs.

See also: https://ytsaurus.tech/docs/en/problems/woodpeckers

◆ UseTableFormats()

TSelf & NYT::TOperationOptions::UseTableFormats ( bool )

Use format from table attribute (for YAMR-like format).

Deprecated

◆ Wait()

TSelf & NYT::TOperationOptions::Wait ( bool value )

inline

Wait for operation finish synchronously.

Deprecated: Use StartOperationMode() instead.

The documentation for this struct was generated from the following file:

yt/cpp/mapreduce/interface/operation.h

Public Types

Public Member Functions