Deepseek's new architecture is insane

Deepseek V4.1 Flash explained #ai #llm #ainews #agi #singularity Thanks to our sponsor Luma. Try it today https://lumalabs.ai/aisearch-sep3

Deepseek V4.1 Flash tech report https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash/blob/main/DeepSeek_V41_Tech_Report.pdf

Also see:
How AI models work: https://youtu.be/U2hZFMVNSE0
DSpark https://youtu.be/J0D7qV3nl7w
Deepseek V4 explained https://youtu.be/XJUpuOBpT-4

00:00 Deepseek V4.1 intro
00:44 The situation
01:28 Prefill vs decode
03:20 The problem
06:28 Causal encoder decoder
08:58 Global vs local cache
09:33 Sliding window attention
11:33 Luma
13:00 Insane cache compression
14:28 CSA2
16:50 Hierarchical sparse indexer
18:20 Local memory problem
19:46 SWA bounded replay
22:30 Single pass mHC
23:48 Engram
25:16 DSpark
26:56 They flattened the curve
28:15 Frontier performance
29:33 Summary

Newsletter: https://aisearch.substack.com/
Find AI tools & jobs: https://ai-search.io/
Support: https://ko-fi.com/aisearch

Here's my equipment, in case you're wondering:
Lenovo Thinkbook: https://amzn.to/4jWeKwH
Dell Precision 5690: https://www.dell.com/en-us/dt/ai-technologies/index.htm?utm_source=AISearchTools&utm_medium=youtube&utm_campaign=precisionai#tab0=0
GPU: Nvidia RTX 5000 Ada https://nvda.ws/3zfqGqS
Mic: Shure SM7B https://amzn.to/3DErjt1
Audio interface: Scarlett Solo https://amzn.to/3qELMeu