NVIDIA DeepSeek R1 FP4 - A quantized version of the DeepSeek R1 model optimized for efficiency and cost reduction.

## Overview of NVIDIA DeepSeek R1 FP4 NVIDIA DeepSeek R1 FP4 is a quantized version of the DeepSeek R1 model, designed to enhance AI model efficiency through FP4 precision. It is optimized for inference performance, reducing operational costs while maintaining high accuracy, making it suitable for both commercial and non-commercial use. ## Overview of NVIDIA DeepSeek R1 FP4 NVIDIA DeepSeek R1 FP4 features include: - **Architecture**: Transformers-based, with the DeepSeek R1 network architecture. - **Quantization**: Reduced to FP4 precision, decreasing bits per parameter from 8 to 4, and reducing disk and GPU memory by approximately 1.6 times. - **Context Length**: Supports up to 128,000 tokens. - **Software and Hardware**: Supported by the TensorRT-LLM runtime engine, runs on NVIDIA Blackwell hardware, and uses the Linux operating system. - **Optimization**: Optimized for the Blackwell architecture, using FP4 precision to significantly improve inference performance and reduce costs. - **Performance Metrics**: Achieves 99.8% FP8 precision in the MMLU general intelligence benchmark, with inference speed increased by 25 times and costs reduced by 20 times. ## Overview of NVIDIA DeepSeek R1 FP4 NVIDIA DeepSeek R1 FP4 achieves 99.8% FP8 precision in the MMLU general intelligence benchmark. It increases inference speed by 25 times and reduces costs by 20 times, demonstrating significant efficiency improvements while maintaining high accuracy. ## Deployment Requirements for NVIDIA DeepSeek R1 FP4 To deploy NVIDIA DeepSeek R1 FP4, the following requirements must be met: - **Hardware**: 8 NVIDIA B200 GPUs. - **Software**: TensorRT-LLM, built from the latest main branch source code, and the Linux operating system. ## Input and Output Types for NVIDIA DeepSeek R1 FP4 NVIDIA DeepSeek R1 FP4 accepts **input** in the form of text, specifically one-dimensional sequences of strings. The **output** is also text, formatted as one-dimensional sequences of strings. ## Inference Engine for NVIDIA DeepSeek R1 FP4 NVIDIA DeepSeek R1 FP4 uses the TensorRT-LLM inference engine, which is tested on NVIDIA B200 hardware. ## Additional Resources for NVIDIA DeepSeek R1 FP4 Additional resources and documentation for NVIDIA DeepSeek R1 FP4 can be found at the following links: - **Model Page**: [NVIDIA DeepSeek R1 FP4 on Hugging Face](https://huggingface.co/nvidia/DeepSeek-R1-FP4) - **Original Model Card**: [DeepSeek R1 on Hugging Face](https://huggingface.co/deepseek-ai/DeepSeek-R1) - **License**: [MIT License on Hugging Face](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/mit.md) - **TensorRT-LLM GitHub**: [TensorRT-LLM Repository](https://github.com/NVIDIA/TensorRT-LLM) - **Security Vulnerability Reporting**: [NVIDIA Security](https://www.nvidia.com/en-us/support/submit-security-vulnerability/) ### Citation sources: - [NVIDIA DeepSeek R1 FP4](https://huggingface.co/nvidia/DeepSeek-R1-FP4) - Official URL Updated: 2025-03-28