Loading...
Searching...
No Matches
NYT::TMapReduceOperationSpecBase< TDerived > Struct Template Reference
Spec of MapReduce operation. More...
#include <operation.h>
Inheritance diagram for NYT::TMapReduceOperationSpecBase< TDerived >:
Public Member Functions | |
| TSelf & | MapperSpec (TUserJobSpec) |
| Spec of map job. | |
| TSelf & | ReducerSpec (TUserJobSpec) |
| Spec of reduce job. | |
| TSelf & | ReduceCombinerSpec (TUserJobSpec) |
| Spec of reduce combiner. | |
| TSelf & | SortBy (TSortColumns) |
| Columns to sort rows by (must include ReduceBy as prefix). | |
| TSelf & | ReduceBy (TSortColumns) |
| Columns to group rows by. | |
| TSelf & | DataSizePerMapJob (ui64) |
| Recommended number of map jobs to run. | |
| TSelf & | PartitionCount (ui64) |
| Recommended number of intermediate data partitions. | |
| TSelf & | PartitionDataSize (ui64) |
| Recommended size of intermediate data partitions. | |
| TSelf & | IntermediateDataAccount (TString) |
| Account to use for intermediate data. | |
| TSelf & | IntermediateDataReplicationFactor (ui64) |
| Replication factor for intermediate data (1 by default). | |
| TSelf & | DataSizePerSortJob (ui64) |
| Recommended size of data to be passed to a single reduce combiner. | |
| TSelf & | Ordered (bool) |
| Whether to guarantee the order of rows passed to mapper matches the order in the table. | |
| TSelf & | ForceReduceCombiners (bool) |
| Guarantee to run reduce combiner before reducer. | |
| Public Member Functions inherited from NYT::TUserOperationSpecBase< TDerived > | |
| TSelf & | FailOnJobRestart (bool) |
| On any unsuccessful job completion (i.e. abortion or failure) force the whole operation to fail. | |
| TSelf & | StderrTablePath (TYPath) |
| Table to save whole stderr of operation. | |
| TSelf & | CoreTablePath (TYPath) |
| Table to save coredumps of operation. | |
| TSelf & | WaitingJobTimeout (TDuration) |
| How long should the scheduler wait for the job to be started on a node. | |
| Public Member Functions inherited from NYT::TOperationSpecBase< TDerived > | |
| TSelf & | TimeLimit (TDuration) |
| Limit on operation execution time. | |
| TSelf & | Alias (TString) |
| Alias for searching for an operation in the future. | |
| TSelf & | Title (TString) |
| Title to be shown in web interface. | |
| TSelf & | Pool (TString) |
| Pool to be used for this operation. | |
| TSelf & | Weight (double) |
| Weight of operation. | |
| FLUENT_OPTIONAL_VECTOR_FIELD_ENCAPSULATED (TString, PoolTree) | |
| Pool tree list that operation will use. | |
| FLUENT_FIELD_OPTION_ENCAPSULATED (TSchedulerResources, ResourceLimits) | |
| How much resources can be consumed by operation. | |
| TSelf & | MaxFailedJobCount (ui64) |
| How many jobs can fail before operation is failed. | |
| TSelf & | Annotations (TNode) |
| TSelf & | Description (TNode) |
Detailed Description
template<typename TDerived>
struct NYT::TMapReduceOperationSpecBase< TDerived >
struct NYT::TMapReduceOperationSpecBase< TDerived >
Spec of MapReduce operation.
Member Function Documentation
◆ DataSizePerMapJob()
template<typename TDerived>
| TSelf & NYT::TMapReduceOperationSpecBase< TDerived >::DataSizePerMapJob | ( | ui64 | ) |
Recommended number of map jobs to run.
`JobCount' has higher priority than NYT::TMapReduceOperationSpecBase::DataSizePerMapJob. This option only provide a recommendation and may be ignored if conflicting with YT internal limits. */ TSelf& MapJobCount ( ui32 ); ;
/**
Recommended of data size for each map job.
DataSizePerJob has lower priority that NYT::TMapReduceOperationSpecBase::MapJobCount. This option only provide a recommendation and may be ignored if conflicting with YT internal limits.
◆ Ordered()
template<typename TDerived>
| TSelf & NYT::TMapReduceOperationSpecBase< TDerived >::Ordered | ( | bool | ) |
Whether to guarantee the order of rows passed to mapper matches the order in the table.
- See also
- NYT::TMapOperationSpec::Ordered for more info.
The documentation for this struct was generated from the following file:
- yt/cpp/mapreduce/interface/operation.h
