David Lin
Senior Backend Engineer
Specialized in data reliability, scalable pipelines, and engineering efficiency. Passionate about tackling complex architecture bottlenecks in high-caliber teams.
Skills
Languages & Environment
Frameworks & Libs
Databases & Infra
Tools & Automation
AI-Native & Agentic Workflows
Work Experience
- Architecture & Scaling: Designed and implemented a highly modular distributed crawler architecture (WeaverCore), compressing a projected “4-week/1-person” development schedule into 2 hours, and successfully deploying the system on the day of proposal.
- Developer Experience (DX) & CI/CD: Engineered an automated CI/CD pipeline using semantic-release and openapi-ts to dynamically generate and publish TypeScript SDKs directly from Swagger specs, ensuring end-to-end type safety and eliminating manual API integration overhead.
- System Reliability: Architected a Redis-based proxy management system to securely distribute resources across high-concurrency processes, ensuring 99.9% system stability during peak loads.
- Performance Optimization: Designed and implemented HTTP streaming APIs using JSON Lines (NDJSON). Successfully compressed client Time-to-Interactive (TTI) from over 30 minutes to under 1 second under extreme data loads, while reducing server memory usage to an O(1) constant footprint, eliminating backend OOM risks during concurrent requests.
- Maintainability & Refactoring: Refactored core data workflows (Argus-v2), decoupling database write paths from legacy JS crawlers and engineering a TypeScript data processing system with layered architecture (Controller-Service-Model) to optimize testability and scalability.
- Observability & Proactive Alerting: Built a comprehensive system monitoring and proactive alerting infrastructure using Grafana, achieving ~90% alert coverage across critical environments and successfully identifying multiple long-standing anomalies.
- Product Development: Directed the end-to-end development of internal automation tools from 0 to 1, successfully translating complex cross-functional requirements into scalable system designs.
- System Architecture: Architected a highly extensible plugin-based system, enabling seamless new feature deployments without requiring full-system redeployments or downtime.
- Microservices Integration: Built and maintained robust platform tools utilizing a microservices architecture, driving faster iteration cycles and independent service scaling.
- Technical Leadership: Spearheaded comprehensive technical strategy and product design initiatives for an early-stage education-focused technology startup.
- Project Delivery: Led the engineering and product team to successfully secure 1st place in the prestigious Zhongshan Startup Container Competition.
Education
Bachelor of Arts in Political Science
National Cheng Kung University | Tainan, Taiwan
Contact Me
If you are interested in my work, or want to discuss remote opportunities, please feel free to drop me a message!