mirror of
https://github.com/exo-explore/exo.git
synced 2025-12-23 14:17:58 -05:00
exo: handle -c flag for multiprocessing helpers in frozen apps
When Python's multiprocessing spawns child processes on macOS (using the
"spawn" method), it also spawns helper processes like the resource tracker
by executing:
./frozen_app -c "from multiprocessing.resource_tracker import main; main()"
A frozen PyInstaller app doesn't understand `-c` natively - it just runs
main(). This causes the resource tracker to fail silently.
This adds a minimal `-c` handler that intercepts the flag, extracts the
inline code, and exec()s it before main() runs. This is required for the
Process() spawn in runner_supervisor.py to work correctly in the DMG.
Note that the pyinstaller docs say `freeze_support` is supposed to make
this work, but it doesn't.
Test plan:
Hardware setup: 3x Mac Studio M3 Ultra connected all-to-all with TB5
- Built a DMG[0].
- Installed on the Macs.
- Started an instance. Got an error this time in ~/.exo/exo.log. The
last DMG from main doesn't show anything when an instance starts, this
now shows the errors.
[0] https://github.com/exo-explore/exo/actions/runs/20464409279/job/58804485197
This commit is contained in:
@@ -1,7 +1,39 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import sys
|
||||
from collections.abc import Sequence
|
||||
from multiprocessing import freeze_support
|
||||
from typing import Final
|
||||
|
||||
from exo.main import main
|
||||
|
||||
INLINE_CODE_FLAG: Final[str] = "-c"
|
||||
|
||||
|
||||
def _maybe_run_inline_code(argv: Sequence[str]) -> bool:
|
||||
"""
|
||||
Reproduce the bare minimum of Python's `-c` flag so multiprocessing
|
||||
helper processes (for example the resource tracker) can execute.
|
||||
"""
|
||||
|
||||
try:
|
||||
flag_index = argv.index(INLINE_CODE_FLAG)
|
||||
except ValueError:
|
||||
return False
|
||||
|
||||
code_index = flag_index + 1
|
||||
if code_index >= len(argv):
|
||||
return False
|
||||
|
||||
inline_code = argv[code_index]
|
||||
sys.argv = ["-c", *argv[code_index + 1 :]]
|
||||
namespace: dict[str, object] = {"__name__": "__main__"}
|
||||
exec(inline_code, namespace, namespace)
|
||||
return True
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
if _maybe_run_inline_code(sys.argv):
|
||||
sys.exit(0)
|
||||
freeze_support()
|
||||
main()
|
||||
|
||||
Reference in New Issue
Block a user