← All work

Open-source / Automation / Production service

Automated scraping, real-time connectivity probing, and high-availability proxy delivery.

High-throughput pipeline that continuously scrapes public network proxies from open web sources, tests real-time latency and connection survival, filters degraded endpoints, and delivers optimized subscription feeds through Nginx.

My role
DevOps & Infrastructure Engineer
Period
2026
Key technologies
Python · Nginx · Ubuntu Server · Bash · Systemd · Asynchronous I/O · SSL/TLS
ProxyPulse03 / SYSTEMS
Scraper → Multi-threaded Probe → Nginx Edge CacheConcept illustration

01 / Context

The problem.

Publicly distributed network proxies are notoriously volatile — suffering from rapid expiration, unpredictable latency, broken routing paths, and massive duplication across public sources.

02 / My contribution

What I built.

Engineered an automated aggregation and testing engine. The system scrapes hundreds of open web sources at scheduled intervals, performs multi-threaded socket connectivity and latency probes to weed out dead or sluggish nodes, removes duplicates, and generates clean, optimized subscription feeds hosted behind a cached Nginx reverse proxy.

SYSTEM PIPELINE ARCHITECTURE

ProxyPulse Real-Time Pipeline Architecture

01

Protocol Scrapers

Vmess / Vless / Shadowsocks

02

Health Probes

Concurrent RTT Validation

03

In-Memory Tier

Dragonfly / Redis Cache

04

Edge Gateway

Low-Latency Subscriptions

03 / Approach

Engineering decisions.

  1. Concurrent asynchronous probing

    Multi-threaded socket and handshake verification tests hundreds of endpoints per minute without blocking the aggregation loop.

  2. Dynamic scoring & deduplication

    Analyzes routing attributes, deduplicates identical IP/port signatures, and ranks nodes based on packet round-trip time (RTT).

  3. Resilient edge delivery via Nginx

    Nginx micro-caching serves valid subscription payloads instantly during peak client updates while background daemons refresh the active pool.

  4. Automated SSL and zero-reload rotation

    Integrated automated certificate provisioning and signal-based daemon updates to eliminate configuration delivery disruptions.

04 / Outcome & context

What this work shows.

Maintained automated 24/7 endpoint quality testing and delivery, ensuring clients only receive verified, responsive network routes.

Automated health probing and edge caching turn volatile public endpoints into reliable, self-healing routing feeds.

Operational project / The cover is a concept illustration. This case study focuses on my engineering contribution and system-level architecture.

Keep exploring

Zero-Trust Cloud Infrastructure