NYT::TMapReduceOperationSpec Struct Reference

Spec of MapReduce operation. More...

#include <operation.h>

Inheritance diagram for NYT::TMapReduceOperationSpec:

Public Member Functions
TSelf &	MapperFormatHints (TUserJobFormatHints)
	Format hints for mapper.
TSelf &	ReducerFormatHints (TUserJobFormatHints)
	Format hints for reducer.
TSelf &	ReduceCombinerFormatHints (TUserJobFormatHints)
	Format hints for reduce combiner.
Public Member Functions inherited from NYT::TMapReduceOperationSpecBase< TMapReduceOperationSpec >
TSelf &	MapperSpec (TUserJobSpec)
	Spec of map job.
TSelf &	ReducerSpec (TUserJobSpec)
	Spec of reduce job.
TSelf &	ReduceCombinerSpec (TUserJobSpec)
	Spec of reduce combiner.
TSelf &	SortBy (TSortColumns)
	Columns to sort rows by (must include ReduceBy as prefix).
TSelf &	ReduceBy (TSortColumns)
	Columns to group rows by.
TSelf &	DataSizePerMapJob (ui64)
	Recommended number of map jobs to run.
TSelf &	PartitionCount (ui64)
	Recommended number of intermediate data partitions.
TSelf &	PartitionDataSize (ui64)
	Recommended size of intermediate data partitions.
TSelf &	IntermediateDataAccount (TString)
	Account to use for intermediate data.
TSelf &	IntermediateDataReplicationFactor (ui64)
	Replication factor for intermediate data (1 by default).
TSelf &	DataSizePerSortJob (ui64)
	Recommended size of data to be passed to a single reduce combiner.
TSelf &	Ordered (bool)
	Whether to guarantee the order of rows passed to mapper matches the order in the table.
TSelf &	ForceReduceCombiners (bool)
	Guarantee to run reduce combiner before reducer.
Public Member Functions inherited from NYT::TUserOperationSpecBase< TMapReduceOperationSpec >
TSelf &	FailOnJobRestart (bool)
	On any unsuccessful job completion (i.e. abortion or failure) force the whole operation to fail.
TSelf &	StderrTablePath (TYPath)
	Table to save whole stderr of operation.
TSelf &	CoreTablePath (TYPath)
	Table to save coredumps of operation.
TSelf &	WaitingJobTimeout (TDuration)
	How long should the scheduler wait for the job to be started on a node.
Public Member Functions inherited from NYT::TOperationSpecBase< TMapReduceOperationSpec >
TSelf &	TimeLimit (TDuration)
	Limit on operation execution time.
TSelf &	Alias (TString)
	Alias for searching for an operation in the future.
TSelf &	Title (TString)
	Title to be shown in web interface.
TSelf &	Pool (TString)
	Pool to be used for this operation.
TSelf &	Weight (double)
	Weight of operation.
	FLUENT_OPTIONAL_VECTOR_FIELD_ENCAPSULATED (TString, PoolTree)
	Pool tree list that operation will use.
	FLUENT_FIELD_OPTION_ENCAPSULATED (TSchedulerResources, ResourceLimits)
	How much resources can be consumed by operation.
TSelf &	MaxFailedJobCount (ui64)
	How many jobs can fail before operation is failed.
TSelf &	Annotations (TNode)
TSelf &	Description (TNode)
Public Member Functions inherited from NYT::TOperationIOSpec< TMapReduceOperationSpec >
TMapReduceOperationSpec &	AddInput (const TRichYPath &path)
TMapReduceOperationSpec &	AddStructuredInput (TStructuredTablePath path)
TMapReduceOperationSpec &	SetInput (size_t tableIndex, const TRichYPath &path)
TMapReduceOperationSpec &	AddOutput (const TRichYPath &path)
TMapReduceOperationSpec &	AddStructuredOutput (TStructuredTablePath path)
TMapReduceOperationSpec &	SetOutput (size_t tableIndex, const TRichYPath &path)
TMapReduceOperationSpec &	AddProtobufInput_VerySlow_Deprecated (const TRichYPath &path)
TMapReduceOperationSpec &	AddProtobufOutput_VerySlow_Deprecated (const TRichYPath &path)
Public Member Functions inherited from NYT::TOperationInputSpecBase
template<class T>
void	AddInput (const TRichYPath &path)
	Add input table path to input path list and specify type of rows.
void	AddStructuredInput (TStructuredTablePath path)
	Add input table path as structured paths.
template<class T>
void	SetInput (size_t tableIndex, const TRichYPath &path)
	Set input table path and type.
const TVector< TStructuredTablePath > &	GetStructuredInputs () const
	Get all input structured paths.
Public Member Functions inherited from NYT::TOperationOutputSpecBase
template<class T>
void	AddOutput (const TRichYPath &path)
	Add output table path to output path list and specify type of rows.
void	AddStructuredOutput (TStructuredTablePath path)
	Add output table path as structured paths.
template<class T>
void	SetOutput (size_t tableIndex, const TRichYPath &path)
	Set output table path and type.
const TVector< TStructuredTablePath > &	GetStructuredOutputs () const
	Get all output structured paths.
Public Member Functions inherited from NYT::TIntermediateTablesHintSpec< TMapReduceOperationSpec >
TMapReduceOperationSpec &	HintMapOutput ()
	Specify intermediate map output type.
TMapReduceOperationSpec &	HintReduceCombinerInput ()
	Specify reduce combiner input.
TMapReduceOperationSpec &	HintReduceCombinerOutput ()
	Specify reduce combiner output.
TMapReduceOperationSpec &	HintReduceInput ()
	Specify reducer input.
TMapReduceOperationSpec &	AddMapOutput (const TRichYPath &path)
	Add output of map stage.
const TVector< TStructuredTablePath > &	GetStructuredMapOutputs () const
const TMaybe< TTableStructure > &	GetIntermediateMapOutputDescription () const
const TMaybe< TTableStructure > &	GetIntermediateReduceCombinerInputDescription () const
const TMaybe< TTableStructure > &	GetIntermediateReduceCombinerOutputDescription () const
const TMaybe< TTableStructure > &	GetIntermediateReducerInputDescription () const

Additional Inherited Members
Public Attributes inherited from NYT::TOperationInputSpecBase
TVector< TRichYPath >	Inputs_
	All input paths.
Public Attributes inherited from NYT::TOperationOutputSpecBase
TVector< TRichYPath >	Outputs_
	All output paths.
Public Attributes inherited from NYT::TIntermediateTablesHintSpec< TMapReduceOperationSpec >
TVector< TRichYPath >	MapOutputs_

Detailed Description

Spec of MapReduce operation.

See also

https://ytsaurus.tech/docs/en/user-guide/data-processing/operations/mapreduce

---

The documentation for this struct was generated from the following file:

• yt/cpp/mapreduce/interface/operation.h