## **Major Features and Improvements** ### **Embedding** - Refine KVInterface::GetShardedSnapshot API. - Undefine EV GPU interface in CPU compile. - Make Embedding backward compatible with previous saved_model. - Log error when EV has been initialized in EV Import OP. ### **Op Implement** - Implement of SliceSend/SliceRecv Op. - Implement FileSliceSend/FileSliceRecvOp. ### **SDK** - Add build SDK package. ### **BugFix** - Fix shared embedding frequency counting problem. - Fix Graph contains EmbeddingVariable compiling issue. - Fix a scheduling issue. - Fix tensor shape meta-data bug for DataFrame Value. ### **ModelZoo** - Set Saver's parameter sharded=True in distributed training. More details of features: [https://deeprec.readthedocs.io/zh/latest/](url) ## **Release Images** ### **CPU Image** `alideeprec/deeprec-release:deeprec2402-cpu-py38-ubuntu20.04` ### **GPU Image** `alideeprec/deeprec-release:deeprec2402-gpu-py38-cu116-ubuntu20.04`

## **Major Features and Improvements** ### **Embedding** - Refactor the data structure of EmbeddingVariable. - Add interface of EmbeddingVar for Elastic Training. - Add GetSnapshot and Create API for EmbeddingVariable. - Remove the dependency on private header file in EmbeddingVariable. ### **Runtime Optimization** - Canonicalize SaveV2 Op device spec in distributed training. - Update log level in direct_session. ### **Distributed** - Add elastic-grpc server. ### **BugFix** - Fix missing return value of RestoreSSD of DramSSDHashStorage. - Fix incorrect frequency in shared-embedding. - Fix set initialized flag too early in restore subgraph. - Fix wgrad bug in Sparse Operation Kit. - Fix hang bug for async embedding lookup. - Fix ps address list sort by index. - Fix SharedEmbeddingColumn with PartitionedEmbedingVariable shape validation error. More details of features: [https://deeprec.readthedocs.io/zh/latest/](url) ## **Release Images** ### **CPU Image** `alideeprec/deeprec-release:deeprec2310-cpu-py38-ubuntu20.04` ### **GPU Image** `alideeprec/deeprec-release:deeprec2310-gpu-py38-cu116-ubuntu20.04`

## **Major Features and Improvements** ### **Embedding** - Support StaticGPUHashMap to optimize EmbeddingVariable in inference. - Update logic of GroupEmbedding in feature_column API. - Refine APIs for foward-backward optimization. - Move insertions of new features into the backward process when lti-tier storage. - Move insertion of new features into the backward ops. - Modify calculation logic of embedding lookup sparse combiner. - Add memory and performance tests of EmbeddingVariable. ### **Graph & Grappler Optimization** - Support IteratorGetNext for SmartStage as a starting node for searching. - Reimplement PrefetchRunner in C++. ### **Runtime Optimization** - Dispatch expensive ops via multiple threads in theadpool. - Enable multi-stream in session_group by default. - Support for loading saved_model with device information when use p and multi_stream. - Make ARENA_ARRAY_SIZE to be configurable. - Optimize EV allocator performance. - Integrate HybridBackend in collective training mode. ### **Ops & Hardware Acceleration** - Disable MatMul fused with LeakyRule when MKL is disabled. ### **Serving** - Clear virtual_device configurations before load new checkpoint. ### **Environment & Build** - Update docker images in user documents. - Update DEFAULT_CUDA_VERSION and DEFAULT_CUDNN_VERSION in configure.py. - Move thirdparties from WORKSPACE to workspace.bzl. - Update urls corresponding to colm, ragel, aliyun-oss-sdk and uuid. - Update default TF_CUDA_COMPUTE_CAPABILITIES to 7.0,7.5,8.0,8.6. - Update SparseOperationKit to v23.5.01 and docker file. ### **BugFix** - Fix issue of missing params while constructing the ngScope. - Fix memory leak to avoid OOM. - Fix shape validation in API shared_embedding_columns. - Fix the device placement bug of stage_subgraph_on_cpu in distributed. - Fix hung issue when using both SOK and SmartStaged simultaneously. - Fix bug: init global_step before saving variables - Fix bug: reserve input nodes, clear saver devices on demand. - Fix memory leak when a graph node is invalid. ### **ModelZoo** - Add examples and docs to demonstrate Collective Training. - Update documents and config files for modelzoo benchmark. - Update modelzoo README. ### **Tool & Documents** - Update cases of configure TF_CUDA_COMPUTE_CAPABILITIES for H100. - Update COMMITTERS.md. - Update device placement documents. - Update document for SmartStage. - Update session_group documents. - Update the download link of the library that Processor depends on. - Update sok to 1.20. More details of features: [https://deeprec.readthedocs.io/zh/latest/](url) ## **Release Images** ### **CPU Image** `alideeprec/deeprec-release:deeprec2306-cpu-py38-ubuntu20.04` ### **GPU Image** `alideeprec/deeprec-release:deeprec2306-gpu-py38-cu116-ubuntu20.04`

## **Major Features and Improvements** ### **Embedding** - Suport tf.int32 dtype using feature_column API `tf.feature_column.categorical_column_with_embedding`. - Make the rules of export frequencies and versions the same as the rule of export keys. - Optimize cuda kernel implementation in GroupEmbedding. - Support to read embedding files with mmap and madvise, and direct IO. - Add double check in find_wait_free of lockless dense hashmap. - Change Embedding init value of version in EV from 0 to -1. - Interface 'GetSnapshot()' backward compatibility. - Implement CPU GroupEmbedding lookup sparse Op. - Make GroupEmbedding compatible with sequence feature_column interface. - Fix sp_weights indices calculation error in GroupEmbedding. - Add group_strategy to control parallelism of group_embedding. ### **Graph & Grappler Optimization** - Support SparseTensor as placeholder in Sample-awared Graph Compression. - Add Dice fusion grappler and ops. - Enable MKL Matmul + Bias + LeakyRelu fusion. ### **Runtime Optimization** - Avoid unnecessary polling in EventMgr. - Reduce lock cost and memory usage in EventMgr when use multi-stream. ### **Ops & Hardware Acceleration** - Register GPU implementation of int64 type for Prod. - Register GPU implementation of string type for Shape, ShapeN and ExpandDims. - Optimize list of GPU SegmentReductionOps. - Optimize zeros_like_impl by reducing calls to convert_to_tensor. - Implement GPU version of SparseSlice Op. - Delay Reshape when rank > 2 in keras.layers.Dense so that post op can be fused with MatMul. - Implement setting max_num_threads hint to oneDNN at compile time. - Implement TensorPackTransH2DOp to improve SmartStage performance on GPU. ### **IO** - Add tensor shape meta-data support for ParquetDataset. - Add arrow BINARY type support for ParquetDataset. ### **Serving** - Add Dice fusion to inference mode. - Enable INFERENCE_MODE in processor. - Support TensorRT 8.x in Inference. - Add configure filed to control enable TensorRT or not. - Add flag for device_placement_optimization. - Avoid to clustering feature column related nodes when enable TensorRT. - Optimize inference latency when load increment checkpoint. - Optimize performance via only place TensorRT ops to gpu device. ### **Environment & Build** - Support CUDA 12. - Update DEFAULT_CUDA_VERSION and DEFAULT_CUDNN_VERSION in configure.py. - Move thirdparties from WORKSPACE to workspace.bzl. - Update urls corresponding to colm, ragel, aliyun-oss-sdk and uuid. ### **BugFix** - Fix constant op placing bug for device placement optimization. - Fix Nan issue occurred in group_embedding API. - Fix SOK not compatible with variable issue. - Fix memory leak when update full model in serving. - Fix 'cols_to_output_tensors' not setted issue in GroupEmbedding. - Fix core dump issue about saving GPU EmbeddingVariable. - Fix cuda resource issue in KvResourceImportV3 kernel. - Fix loading signature_def with coo_sparse bug and add UT. - Fix the bug that the training ends early when the workqueue is enabled. - Fix the control edge connection issue in device placement optimization. ### **ModelZoo** - Modify GroupEmbedding related function usage. - Update masknet example with layernorm. ### **Tool & Documents** - Add tools for remove filtered features in checkpoint. - Add Arm Compute Library (ACL) user documents. - Update Embedding Variable document to fix initializer config example. - Update GroupEmbedding document. - Update processor documents. - Add user documents for intel AMX. - Add TensorRT usage documents. - Update documents for ParquetDataset. More details of features: [https://deeprec.readthedocs.io/zh/latest/](url) ## **Release Images** ### **CPU Image** `alideeprec/deeprec-release:deeprec2304-cpu-py38-ubuntu20.04` ### **GPU Image** `alideeprec/deeprec-release:deeprec2304-gpu-py38-cu116-ubuntu20.04`

## **Major Features and Improvements** ### **Embedding** - Support same saver graph for EmbeddingVariable on GPU/CPU devices. - Support save and restore parameters in HBM storage of EmbeddingVariable. - Add GPU apply ops of Adam, AdamAsync, AdamW for multi-tier storage of EmbeddingVariable. - Place output of KvResourceIsInitializedOp on CPU. - Support GroupEmbedding to pack multiple feature columns lookup/apply. - Optimize HBM-DRAM storage of EmbeddingVariable with intra parallelism and fine-grained synchronization. - Support not saving filtered features when saving checkpoint. - Support localized mode fusion in GroupEmbedding. - Support to avoid preloaded IDs being eliminated in multi-tier embedding's cache. - Support COMPACT layout to reduce memory cost in EmbeddingVariable. - Support to ignore version when restore Embedding Variable with TF_EV_RESET_VERSION. - Support restore custom dimension of Embedding Variable. - Support merge and delete checkpoint files of SSDHash storage. ### **Graph & Grappler Optimization** - Optimize SmartStage by prefetching LookupID op. - Decouple SmartStage and forward backward joint optimization. - Support Sample-awared Graph Compression. - Support CUDA multi-stream for Stage. - Improve Device Placement Optimization performance. - Add TensorBufferPutGpuOp to improve SmartStage performance on GPU device. ### **Runtime Optimization** - Enable EVAllocator by default. - Optimize executor to eliminate sort latency and reduce memory. ### **Ops & Hardware Acceleration** - Add list of GPU Ops for forward backward joint optimization. - Optimize FusedBatchNormGrad on CPU device. - Support NCHW format input for FusedBatchNormOp. - Use new asynchronous evaluation in Eigen to FusedBatchNorm. - Add exponential_avg_factor attribute to FusedBatchNorm* kernels. - Change AliUniqueGPU kernel implementation to AsyncOpKernel. - Support computing exponential running mean and variance in fused_batch_norm. - Upgrade oneDNN to 2.7 and ACL to 22.08. - Use global cache for MKL primitives for ARM. - Disable optimizing batch norm as sequence of post ops on AArch64. - Restore re-mapper and fix BatchMatmul and FactoryKeyCreator under AArch64 + ACL. ### **Distributed** - Speedup SOK by GroupEmbedding which fuse multiple feature column together. ### **Serving** - Support to setup gpu config in SessionGroup. - Support to use multiple GPUs in SessionGroup. - Support processor to set multi-stream option. - Add flag to disable per_session_host_allocator. - Run init_op on all sessions in session_group. - Skip invalid request and return error msg to client. - Use graph signature as the key to get runtime executor. ### **Environment & Build** - Optimize compile time for kv_variable_ops module. - Add dataset headers for custom op compilation. - Add docker images for ARM based on ubuntu22.04. - Upgrade BAZEL version to 3.7.2. ### **BugFix** - Do not cudaSetDevice to invisible GPU in CreateDevices. - Fix concurrency issue caused by not reference to same lock in multi-tier storage. - Fix parse input request bug. - Fix the bug when saving empty GPU EmbeddingVariable. - Fix the concurrency issue between feature eviction and embedding lookup in asynchronous training. ### **ModelZoo** - Support Parquet Dataset in list of models. - Add GPU benchmark in Modelzoo. - Unify the usage of price column in Taobao dataset. - Add DeepFM model with int64 categorical id input. - Update dataset url in Modelzoo. ### **Tool & Documents** - Add checkpoint meta transformer tool. - Add list of user documents in English. More details of features: [https://deeprec.readthedocs.io/zh/latest/](url) ## **Release Images** ### **CPU Image** `alideeprec/deeprec-release:deeprec2302-cpu-py38-ubuntu20.04` ### **GPU Image** `alideeprec/deeprec-release:deeprec2302-gpu-py38-cu116-ubuntu20.04`

## **Major Features and Improvements** ### **Embedding** - Support same saver graph for EmbeddingVariable on GPU/CPU devices. - Support save and restore parameters in HBM storage of EmbeddingVariable. - Add GPU apply ops of Adam, AdamAsync, AdamW for multi-tier storage of EmbeddingVariable. - Place output of KvResourceIsInitializedOp on CPU. - Support GroupEmbedding to pack multiple feature columns lookup/apply. - Optimize HBM-DRAM storage of EmbeddingVariable with intra parallelism and fine-grained synchronization. - Support not saving filtered features when saving checkpoint. - Support localized mode fusion in GroupEmbedding. - Support to avoid preloaded IDs being eliminated in multi-tier embedding's cache. - Support COMPACT layout to reduce memory cost in EmbeddingVariable. - Support to ignore version when restore Embedding Variable with TF_EV_RESET_VERSION. - Support restore custom dimension of Embedding Variable. - Support merge and delete checkpoint files of SSDHash storage. ### **Graph & Grappler Optimization** - Optimize SmartStage by prefetching LookupID op. - Decouple SmartStage and forward backward joint optimization. - Support Sample-awared Graph Compression. - Support CUDA multi-stream for Stage. - Improve Device Placement Optimization performance. - Add TensorBufferPutGpuOp to improve SmartStage performance on GPU device. ### **Runtime Optimization** - Enable EVAllocator by default. - Optimize executor to eliminate sort latency and reduce memory. ### **Ops & Hardware Acceleration** - Add list of GPU Ops for forward backward joint optimization. - Optimize FusedBatchNormGrad on CPU device. - Support NCHW format input for FusedBatchNormOp. - Use new asynchronous evaluation in Eigen to FusedBatchNorm. - Add exponential_avg_factor attribute to FusedBatchNorm* kernels. - Change AliUniqueGPU kernel implementation to AsyncOpKernel. - Support computing exponential running mean and variance in fused_batch_norm. - Upgrade oneDNN to 2.7 and ACL to 22.08. - Use global cache for MKL primitives for ARM. - Disable optimizing batch norm as sequence of post ops on AArch64. - Restore re-mapper and fix BatchMatmul and FactoryKeyCreator under AArch64 + ACL. ### **Distributed** - Speedup SOK by GroupEmbedding which fuse multiple feature column together. ### **Serving** - Support to setup gpu config in SessionGroup. - Support to use multiple GPUs in SessionGroup. - Support processor to set multi-stream option. - Add flag to disable per_session_host_allocator. - Run init_op on all sessions in session_group. - Skip invalid request and return error msg to client. - Use graph signature as the key to get runtime executor. ### **Environment & Build** - Optimize compile time for kv_variable_ops module. - Add dataset headers for custom op compilation. - Add docker images for ARM based on ubuntu22.04. - Upgrade BAZEL version to 3.7.2. ### **BugFix** - Do not cudaSetDevice to invisible GPU in CreateDevices. - Fix concurrency issue caused by not reference to same lock in multi-tier storage. - Fix parse input request bug. - Fix the bug when saving empty GPU EmbeddingVariable. - Fix the concurrency issue between feature eviction and embedding lookup in asynchronous training. ### **ModelZoo** - Support Parquet Dataset in list of models. - Add GPU benchmark in Modelzoo. - Unify the usage of price column in Taobao dataset. - Add DeepFM model with int64 categorical id input. - Update dataset url in Modelzoo. ### **Tool & Documents** - Add checkpoint meta transformer tool. - Add list of user documents in English. More details of features: [https://deeprec.readthedocs.io/zh/latest/](url) ## **Release Images** ### **CPU Image** `alideeprec/deeprec-release:deeprec2302-cpu-py38-ubuntu20.04` ### **GPU Image** `alideeprec/deeprec-release:deeprec2302-gpu-py38-cu116-ubuntu20.04`

## **Major Features and Improvements** ### **BugFix** - Add flag to disable per_session_host_allocator. - Fix bug of saving EmbeddingVariable with int32 type. - Revert "Support fused batchnorm with any ndims and axis". ## **Release Images** ### **CPU Image** `alideeprec/deeprec-release:deeprec2212u1-cpu-py38-ubuntu20.04` ### **GPU Image** `alideeprec/deeprec-release:deeprec2212u1-gpu-py38-cu116-ubuntu20.04`

## **Major Features and Improvements** ### **BugFix** - Add flag to disable per_session_host_allocator. - Fix bug of saving EmbeddingVariable with int32 type. - Revert "Support fused batchnorm with any ndims and axis". ## **Release Images** ### **CPU Image** `alideeprec/deeprec-release:deeprec2212u1-cpu-py38-ubuntu20.04` ### **GPU Image** `alideeprec/deeprec-release:deeprec2212u1-gpu-py38-cu116-ubuntu20.04`