How to avoid overloading shared hosting — PIDs, workers, cron and process supervision.

The problem was not one worker. It was the absence of one process-ownership model.

Several independent execution mechanisms were active in the same environment. Some workers had their own work cycle, cron launched other jobs, a watchdog reacted to service state, a supervisor kept processes alive and an orchestrator started additional system components.

Each mechanism was reasonable on its own. Looking at the whole environment showed that the same function could be started more than once. If an earlier instance had not finished before the next trigger, another instance appeared; child processes accelerated PID growth further.

A PID alone is not enough

A process number confirms an entry in the process table, but it does not describe the role, health or lifecycle ownership. Before restarting anything, supervision should verify process identity, health, the child-process tree and the mechanism authorised to manage that role.

Architecture decision: 1 role = 1 process owner = 1 start mechanism

Before adding a worker, check whether the function already exists. If it does, update the current mechanism or replace it. Do not start a new version next to an old one merely because the previous instance is assumed to have stopped.

Use a queue or explicit parallelism

If two tasks should not run at the same time, put them behind one queue. If parallel execution is required, make it part of the architecture: each domain gets an explicit worker, responsibility and process budget.

For frequent cycles, prefer a worker to repeated cron starts

For frequently repeated work, one long-running worker can be a better model: run → wait → run → healthcheck. Cron remains appropriate for genuinely periodic jobs such as backups, housekeeping, daily reports or audits.

A watchdog should diagnose before restarting

A missing signal does not necessarily mean that the process is gone. Before a restart, check the main process, child processes, health state, service owner and whether another mechanism is supervising the same role.

Result: fewer processes were a consequence; predictability was the goal

After reorganising workers, start mechanisms and supervision, the active process count fell from about 100 to 48. More importantly, the conditions that had allowed the environment to escalate into hundreds of PIDs were constrained. The reduction is not a universal benchmark; it describes this specific environment and incident.

Shared-hosting blast radius

When several projects share one resource budget, process or memory growth in one application can affect other services. Monitoring therefore needs to cover the account as a whole: user PIDs, RAM, process trees, active schedules and supervision mechanisms.

Six design rules

1. Check whether the role already exists before adding another implementation.
2. One worker should have one lifecycle owner.
3. Design parallel execution explicitly.
4. Cron is not a supervisor.
5. A watchdog diagnoses before restarting.
6. Monitor the environment, not only one service.

FAQ