Compare commits

..

32 Commits

Author SHA1 Message Date
29ea82821d fix(naver-fetch): ingest 응답 필드 matched→queued (BE async 8e28ce9 정합) 2026-07-10 11:26:12 +09:00
8e64c4feb8 fix(naver-fetch): realestate base :18800 + 동별 ingest + 180s no-retry 2026-07-10 10:55:51 +09:00
0b4e4c2e0e feat(naver-fetch): Dockerfile(Playwright) + compose 서비스(18716) + .env.example 2026-07-10 10:21:54 +09:00
56c13cbcbd feat(naver-fetch): FastAPI lifespan + heartbeat 배선 + /health 2026-07-10 10:21:04 +09:00
e3c7f24794 feat(naver-fetch): fetch_worker (run_cycle/loop/heartbeat) 2026-07-10 10:21:03 +09:00
03deb12bd1 feat(naver-fetch): NaverClient (Playwright 토큰 + httpx fetch) 2026-07-10 10:21:03 +09:00
b0155830f2 feat(naver-fetch): NAS realestate 클라이언트 (targets/ingest) 2026-07-10 10:21:03 +09:00
6b0e91342c feat(naver-fetch): 스캐폴딩 + config 2026-07-10 10:21:03 +09:00
9abca3eeab docs: /infra 워커 관측 규칙 + trade-monitor climax 정합 반영
- CLAUDE.md: 모든 WSL docker 워커 /infra 관측 필수 규칙(BE 팀규칙) +
  services 행에 trade-monitor(:18715) 반영
- README.md: sell_climax 정합·36 tests·env 우선순위 문구 갱신

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N83vbXEA8h83GMXQcg8fxD
2026-07-03 11:15:36 +09:00
5dbb11ac83 fix(trade-monitor): sell_climax holdings_intel 정합
BE 회신(holdings_intel.py:109-118)에 맞춰 반전 기준을
price<day_open → price<day_high×climax_close_pct(윗꼬리)로 변경.
- kis_client.get_quote에 day_high(stck_hgpr) 추가
- monitor._build_ctx가 day_high를 ctx로 전달
- climax_vol_x·climax_close_pct를 monitor-set exit_params에서 읽기
  (fallback: TM_CLIMAX_VOL_MULT/0.97)
- 테스트 36/36 (climax exit_params 2건 추가)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N83vbXEA8h83GMXQcg8fxD
2026-07-03 11:15:27 +09:00
8e1b20190d docs(readme): trade-monitor 워커 섹션 + /infra 관측 규칙 반영 2026-07-03 11:05:57 +09:00
fa6ef6c5c8 feat(trade-monitor): Dockerfile + compose 서비스(18715) + .env.example 2026-07-03 01:48:14 +09:00
12aa55ed14 feat(trade-monitor): FastAPI lifespan + heartbeat 배선 + /health 2026-07-03 01:47:34 +09:00
ce8983c1b9 feat(trade-monitor): monitor 오케스트레이션 (run_cycle/loop/state_fn) 2026-07-03 01:47:34 +09:00
04aff34883 feat(trade-monitor): NAS trade-alert 클라이언트 (monitor-set/report) 2026-07-03 01:46:37 +09:00
d761716e00 feat(trade-monitor): KIS 자체 토큰 + quote + 일봉 클라이언트 2026-07-03 01:46:37 +09:00
241ce41a6a feat(trade-monitor): 매수/매도 조건 로직 (§6 8개 조건) 2026-07-03 01:45:41 +09:00
366a9160d5 feat(trade-monitor): 순수 지표 모듈 (sma/rsi/highest_high) 2026-07-03 01:45:41 +09:00
141209ad42 feat(trade-monitor): 스캐폴딩 + config 2026-07-03 01:44:25 +09:00
03e50d2be1 fix(task-watcher): _shared를 빌드 컨텍스트에 포함 (heartbeat import 크래시 수정)
task-watcher는 build context가 ./task-watcher라 services/_shared가 이미지에
없었음 → A3가 추가한 `from _shared.heartbeat import` ModuleNotFoundError로
컨테이너 즉시 크래시(재시작 후 alive=false). render 워커 패턴대로
context=. + COPY _shared /app/_shared + PYTHONPATH=/app 로 수정.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019LV86jBozkNhSFXJA412fq
2026-07-01 02:26:57 +09:00
54fca07d43 feat(ai_trade): NAS Redis heartbeat (trader market_open/closed)
- ai_trade/heartbeat.py: build_trader_payload() + heartbeat_loop() 자체 미니 헬퍼
  (Windows 호스트 실행이라 _shared import 경로 달라 독립 구현, 계약은 동일)
- ai_trade/main.py: lifespan에 hb_task spawn + shutdown 시 cancel
  state_fn = scheduler._is_market_day & _is_polling_window(KST now) 조합
  signals = len(state.signals) 실시간 주입
- requirements.txt: redis>=5.0 추가
- ai_trade/tests/test_heartbeat.py: build_trader_payload 3케이스 TDD 검증

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019LV86jBozkNhSFXJA412fq
2026-07-01 01:07:00 +09:00
574b5712c3 feat(task-watcher): heartbeat 발신 (state=mode, paused 이유 노출)
- watcher_loop 에서 mode 판정 직후 worker:task-watcher:heartbeat SET EX 45
- payload: build_payload(state=mode, extra={"mode": mode})
- LOOP_INTERVAL 30s < TTL 45s → 만료 전 주기적 갱신
- conftest.py 추가: services/ 를 sys.path에 주입해 _shared import 가능
- tests/test_watcher.py: payload kind/state/mode 필드 검증 (1 passed)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-01 00:59:28 +09:00
2ff31b2e76 feat(render-workers): 4 render 워커 heartbeat 배선 + poll_once 카운터
- services/_shared/heartbeat.py (A1) WorkerStats/utc_now_iso/heartbeat_loop 소비
- image-render / video-render / music-render / insta-render 각 worker.py:
  stats = WorkerStats() 모듈 레벨 추가, poll_once에서 dispatch 전 busy=True,
  ack 후 jobs_done+1 / fail 후 jobs_failed+1 + last_job_at + busy=False
- 각 main.py: lifespan에 aioredis(decode_responses=False) + heartbeat_loop 태스크 spawn,
  종료 시 cancel + aclose
- 각 tests/test_worker.py: test_poll_once_increments_jobs_done 추가
  (image:flux / video:sora / music:suno / insta:_process_one mock)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019LV86jBozkNhSFXJA412fq
2026-07-01 00:52:57 +09:00
d1b9ff570d feat(_shared): 워커 heartbeat 모듈 (worker:<name>:heartbeat TTL SET)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-01 00:43:01 +09:00
4fb3d12244 merge: co-gahusb AI 클라이언트 배선 2026-06-12 23:46:35 +09:00
789a807d50 feat(co-gahusb): AI 클라이언트 배선 (.mcp.json + 역할 블록)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-12 23:46:34 +09:00
ad141a2887 fix(insta-render): INSTA_MEDIA_ROOT를 insta_cards 하위로 정렬 (nginx 서빙 경로 일치)
워커가 INSTA_MEDIA_ROOT/{slate_id}에 PNG를 쓰는데 기본값 /mnt/nas/webpage/data/insta가 insta_cards 서브디렉토리를 누락 → data/insta/{id}에 저장. 그러나 nginx(/media/insta→/data/insta_cards), insta-lab CARDS_DIR, frontend 마운트, 구 렌더는 전부 data/insta/insta_cards/{id}를 기대 → /media/insta/{id}/NN.png 404.

INSTA_MEDIA_ROOT을 /mnt/nas/webpage/data/insta/insta_cards로 정정(.env + compose 기본값 + .env.example). 코드 무변경 → 컨테이너 recreate만으로 적용(rebuild 불요). SMB 볼륨 마운트는 상위 디렉토리라 그대로 유효.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-11 01:18:09 +09:00
6774067505 fix(insta-render): 큐 연결 socket_timeout=30 (None→30 교정)
근본원인 실험 확정: redis-py 블로킹 read에서 socket_timeout이 BLMOVE 블록(5s)
이하/None이면 read_timeout 경계 경합으로 간헐 "Timeout reading" → dequeue 실패
→ 슬레이트 draft 정지. socket_timeout 10/30은 모든 실험에서 안정. 블록보다 큰
30으로 명시(직전 None 커밋은 단독 테스트만 통과시켜 오도 — 재사용 패턴서 깨짐).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-08 03:17:34 +09:00
c451f5313b fix(insta-render): BLMOVE dequeue가 짧은 socket_timeout으로 깨지던 문제 해결
REDIS_URL의 socket_timeout(<5s)이 ReliableQueue BLMOVE 5초 블록보다 짧아
idle dequeue마다 "Timeout reading"으로 잡을 못 꺼내 슬레이트가 draft에 정지(~2026-05-22~).
큐 연결을 socket_timeout=None + socket_keepalive로 생성(make_queue_redis)해 정상화.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-06 16:08:43 +09:00
9241b5cd90 fix(insta-render): fonts.ready 대기 + PNG 비어있음 검증 (렌더 known-issue 해결)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-06 12:53:07 +09:00
8bfc8e153f polish(insta-render): CSS accent | safe + cover sub clamp
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-06 12:50:25 +09:00
232aa52adb feat(insta-render): 모던 미니멀 디자인 시스템 템플릿
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-06 12:46:19 +09:00
70 changed files with 5050 additions and 58 deletions

9
.mcp.json Normal file
View File

@@ -0,0 +1,9 @@
{
"mcpServers": {
"co-gahusb": {
"type": "http",
"url": "https://gahusb.synology.me/api/co/mcp",
"headers": { "Authorization": "Bearer ${CO_BUS_KEY}" }
}
}
}

View File

@@ -15,12 +15,14 @@ Windows AI 머신 (AMD 9800X3D + RTX 5070 Ti 16GB) 의 두 신호 파이프라
| `ai_trade/` | 자동매매 메인 (구 `signal_v2` 2026-05-19 rename) — Chronos-bolt + 분봉 모멘텀 + KIS WebSocket + 신호 생성 | `:8001` | **Phase 4 완료 (2026-05-17)**, Phase 5 대기 |
| `legacy/start_v1.bat` | (deprecated) V1 진입점 — root `start.bat`에서 이동됨. 자동 실행 차단 | — | **OFF** |
| `ai_trade/start.bat` | 자동매매 진입점 | — | `ai_trade/main.py` uvicorn 실행 |
| `services/` | (예정) NAS↔Windows 분산 worker — insta-render·music-render·video-render·task-watcher | 18710~ | **Plan-B-Insta 작업 중** |
| `services/` | NAS↔Windows 분산 worker — insta·music·video·image-render·task-watcher·**trade-monitor(:18715, 실시간 매매 알람)** | 18710~ | 운영 |
| `.env` | 환경변수 (`KIS_REAL_*`, `TELEGRAM_*`, `STOCK_API_URL`, `WEBAI_API_KEY`, `LOG_LEVEL`) | — | |
| `requirements.txt` | 공용 의존성 | — | torch, chronos-forecasting, fastapi, httpx, websockets 등 |
`.venv`**구조적으로 깨짐**: `pyvenv.cfg` 가 한글 사용자 경로(`C:\Users\박재오\...`) 를 포함하여 콘솔 코드페이지가 roundtrip 못함. 테스트는 시스템 Python 으로 실행: `C:\Users\jaeoh\AppData\Local\Programs\Python\Python312\python.exe -m pytest ai_trade/tests -q`.
> **분산 워커 /infra 관측 규칙 (팀 규칙, BE 제정)**: 모든 WSL docker 워커는 ① Redis `worker:<name>:heartbeat`(EX45) 발신 ② BE `node_monitor.WORKER_REGISTRY` 등재 ③ `/api/agent-office/nodes`·web-ui `/infra` 노출이 필수. trade-monitor는 kind=trader로 등재됨.
---
## 서버 시작 방식
@@ -137,3 +139,14 @@ cd C:\Users\jaeoh\Desktop\workspace\web-ai\ai_trade
- **spec amendment 발생 시**: 코드는 `web-ai` 에 commit, spec 갱신은 `web-ui/docs/superpowers/specs/` 에 commit (Phase 4 spread formula 변경 사례 = web-ui commit `534ded5`)
자세한 V1 가이드는 `signal_v1/CLAUDE.md` 참조 (있다면).
---
## 협업 팀 버스 (co-gahusb) — 이 세션의 역할: **AI**
이 세션은 AI 리서치(AI) 역할이다. co-gahusb MCP 툴로 다른 세션(FE/BE/Producer)과 협업한다.
- **소유권**: 이 세션은 `web-ai` repo만 쓴다(FE=web-ui, BE=web-backend).
- **공유 리소스 변경 전 반드시 `acquire_lock(resource, "AI")`**: 대상 = `nas-deploy`, `stock-db-schema`, `lotto-db-schema`, `memory-mirror`, `nginx-conf`, `compose`. 점유 중이면 대기, 긴 작업은 `heartbeat_lock`, 끝나면 `release_lock`.
- **모든 툴 호출에 `role="AI"`** (또는 `from_role`/`created_by`에 AI).
- **수신**: `/loop`로 주기적으로 `read_inbox("AI", after_id=<last>)` + `list_tasks(assignee_role="AI")` 확인.
-`CO_BUS_KEY`는 환경변수로 주입(커밋 금지). `.mcp.json``${CO_BUS_KEY}`가 프로세스 환경변수에서 치환됨 → `setx CO_BUS_KEY "..."` 후 새 터미널에서 `claude` 실행.

View File

@@ -3,7 +3,7 @@
Windows AI 머신(AMD 9800X3D + RTX 5070 Ti 16GB)에서 동작하는 두 영역의 서비스:
1. **ai_trade** — Confidence Signal Pipeline V2. NAS stock 백엔드와 KIS Open API를 결합해 매수/매도 신호를 생성하는 FastAPI 워커.
2. **services** — NAS↔Windows 분산 렌더링 워커(인스타 카드 / 음악 / 영상 / 이미지) + task-watcher.
2. **services** — NAS↔Windows 분산 워커: 렌더링(인스타 카드 / 음악 / 영상 / 이미지) + task-watcher + **trade-monitor**(실시간 매매 알람).
상위 워크스페이스 컨텍스트는 `../CLAUDE.md`, 본 디렉토리 상세는 `CLAUDE.md`, 운영 체크포인트는 `CHECK_POINT.md` 참조.
@@ -20,6 +20,7 @@ Windows AI 머신(AMD 9800X3D + RTX 5070 Ti 16GB)에서 동작하는 두 영역
| `services/video-render/` | sora / veo / kling / seedance 4 provider 영상 생성 게이트웨이. `queue:video-render` 소비. | `:18712` |
| `services/image-render/` | gpt_image / nano_banana / flux(ComfyUI 로컬) 3 provider. `queue:image-render` 소비. | `:18714` |
| `services/task-watcher/` | 박재오 작업 시간대에 `queue:paused` 토글 → 워커 일시 정지. | `:18713` |
| `services/trade-monitor/` | 실시간 매매 알람. monitor-set pull → KIS 시세 + TA 조건(§6 8종) → report + heartbeat(kind=trader). 무상태. | `:18715` |
| `legacy/signal_v1/` | ⚠ **DEPRECATED** (2026-05-19). LSTM 봇. 자동 실행 차단됨. | OFF |
---
@@ -130,6 +131,60 @@ redis-cli -h 192.168.45.54 KEYS 'processing:*'
---
## trade-monitor — 실시간 매매 알람 워커 (신규 2026-07-03)
NAS stock 백엔드(`:18500`)에서 `monitor-set`을 60초마다 pull하고, KIS 실시간/일봉 시세로 TA 조건을 평가해 발화집합을 `report`로 전송. NAS가 edge diff로 신규 알림만 텔레그램/프론트에 노출. **워커 무상태**(dedup은 NAS 영속). 포트 `:18715`, WSL2 Docker. 상세 설계·조건 규칙은 `services/trade-monitor/DESIGN.md`.
### 루프 (1분)
1. `GET /api/webai/trade-alert/monitor-set` (X-WebAI-Key) → buy_targets(watchscreener) + sell_targets(보유 avg_price/qty/holding_high) + buy_params/exit_params + session.
2. `session=="closed"`면 KIS 호출 0, idle. **비-KRX(알파벳) 티커 skip**(워커 책임).
3. KIS quote + 일봉 250봉 → 지표 계산 → 조건 평가(종목 단위 실패 격리).
4. `POST /api/webai/trade-alert/report {as_of, firing:[...]}` — 빈 배열도 전송(edge clear).
5. heartbeat `worker:trade-monitor:heartbeat` EX45 (kind=trader, state=market_open|market_closed|idle + last_alert_at). 60초 루프 > TTL45 만료갭 회피 위해 **15초 독립 태스크**.
### 조건 (§6 — `condition` 문자열이 FE 라벨/뱃지로 그대로 매핑됨)
- **매수**: `buy_ma20_pullback`(정배열 + ma20 근접 반등), `buy_breakout`(20봉 고점 돌파 + 거래량 배수), `buy_rsi_bounce`(RSI 과매도 반등, 무상태).
- **매도**: `sell_stop_loss`, `sell_take_profit`, `sell_trailing_stop`, `sell_ma_break`(ma50/ma200 severity), `sell_climax`(거래량 급증 + `price<day_high×0.97` 윗꼬리 — holdings_intel 정합됨, `exit_params` 파라미터화).
### 핵심 파일
| 파일 | 책임 |
|------|------|
| `config.py` | Settings (`TM_` 접두사, ai_trade와 분리) |
| `indicators.py` | 순수: `sma` / `rsi_series`(Wilder) / `highest_high` |
| `conditions.py` | 순수 §6: `evaluate_buy` / `evaluate_sell` |
| `kis_client.py` | KIS **자체 OAuth 토큰** + `get_quote` + `get_daily_ohlcv` + 0.5s throttle |
| `nas_client.py` | monitor-set / report (X-WebAI-Key + retry) |
| `monitor.py` | `run_cycle` / `monitor_loop` / `make_state_fn` |
| `main.py` | FastAPI lifespan + `_shared.heartbeat_loop` 배선 + `/health` |
### 환경 변수
| 변수 | 기본 | 설명 |
|------|------|------|
| `NAS_BASE_URL` | `http://192.168.45.54:18500` | stock 백엔드 |
| `WEBAI_API_KEY` | (필수) | X-WebAI-Key |
| `REDIS_URL` | `redis://192.168.45.54:6379` | heartbeat |
| `TM_KIS_APP_KEY` / `TM_KIS_APP_SECRET` / `TM_KIS_ACCOUNT` | (필수) | KIS **전용** 자체 토큰(ai_trade와 분리 발급 → 토큰 상호 무효화·EGW00201 회피) |
| `TM_KIS_IS_VIRTUAL` | `0` | 실전/모의 |
| `TM_LOOP_INTERVAL` | `60` | 루프 주기(초) |
| `TM_CLIMAX_VOL_MULT` | `3.0` | sell_climax 거래량 배수 fallback (우선순위: monitor-set `exit_params.climax_vol_x` > 이 값) |
### 상태
⏳ 구현·머지 완료(테스트 36/36, sell_climax holdings_intel 정합 포함), **미배포**. 배포 전: ① 전용 KIS 앱키 발급·주입(박재오 진행 중) ② 첫 운영 KIS 필드 검증(stck_hgpr 등). BE가 `node_monitor.WORKER_REGISTRY`에 등재 완료 → 배포 시 `/api/agent-office/nodes`·web-ui `/infra`에 trader 노드 자동 노출(미배포 동안 down, 무경보).
### 시작 (NAS, WSL2 Docker)
```bash
cd services
docker compose up -d trade-monitor
```
---
## 환경 변수 (ai_trade)
| 변수 | 기본 | 설명 |
@@ -168,6 +223,7 @@ cd services/insta-render && python -m pytest tests/ -q
cd services/music-render && python -m pytest tests/ -q
cd services/video-render && python -m pytest tests/ -q
cd services/image-render && python -m pytest tests/ -q
cd services/trade-monitor && python -m pytest tests/ -q # 36 tests
```
**`.venv` 한글 사용자 경로 깨짐**으로 시스템 Python(`C:\Users\jaeoh\AppData\Local\Programs\Python\Python312\python.exe`) 사용 권장. 또는 `py -3.12 -m pytest …`.
@@ -183,6 +239,8 @@ cd services/image-render && python -m pytest tests/ -q
5. **services worker_id** — env로 명시 권장. hostname 기반 default는 컨테이너 재기동 시 바뀌어 orphan 분실 위험.
6. **dead-letter 누적**`redis-cli LLEN dead_letter:*` 정기 점검 필요.
7. **Dockerfile build context**`services/` 루트 (각 worker 디렉토리 아님). compose 변경 동반.
8. **분산 워커 /infra 관측 필수 (팀 규칙)** — 모든 WSL docker 워커는 heartbeat(`worker:<name>:heartbeat` EX45) + BE `node_monitor.WORKER_REGISTRY` 등재 + `/infra` 노출이 필수. trade-monitor는 kind=trader로 등재됨.
9. **trade-monitor KIS 앱키 분리** — ai_trade와 **다른 전용 app_key**(`TM_KIS_*`) 사용. 같은 app_key 공유 시 토큰 상호 무효화 + EGW00201. 실전 최대 89앱 발급 가능.
---

57
ai_trade/heartbeat.py Normal file
View File

@@ -0,0 +1,57 @@
"""ai_trade heartbeat — NAS Redis로 worker:ai_trade:heartbeat SET.
Global Constraints 계약 1: kind=trader, state=market_open|market_closed.
ai_trade는 Windows 호스트 실행이라 _shared import 경로가 달라 자체 미니 헬퍼로 둔다.
"""
from __future__ import annotations
import asyncio
import datetime as dt
import json
import logging
import os
import redis.asyncio as aioredis
logger = logging.getLogger(__name__)
REDIS_URL = os.getenv("REDIS_URL", "redis://192.168.45.54:6379")
KEY = "worker:ai_trade:heartbeat"
INTERVAL = int(os.getenv("HEARTBEAT_INTERVAL", "15"))
TTL = int(os.getenv("HEARTBEAT_TTL", "45"))
def build_trader_payload(state: str, signals: int = 0) -> str:
"""JSON 문자열 반환. state: 'market_open' | 'market_closed'."""
return json.dumps({
"name": "ai_trade",
"kind": "trader",
"state": state,
"ts": dt.datetime.now(dt.timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ"),
"last_job_at": None,
"jobs_done": signals,
"jobs_failed": 0,
})
async def heartbeat_loop(state_fn) -> None:
"""Redis에 HEARTBEAT_INTERVAL마다 SET EX TTL.
Args:
state_fn: () -> (state: str, signals: int). 호출자가 폴링 윈도우 판정 주입.
"""
redis = aioredis.from_url(REDIS_URL, decode_responses=False)
try:
while True:
try:
state, signals = state_fn()
payload = build_trader_payload(state, signals)
await redis.set(KEY, payload, ex=TTL)
logger.debug("ai_trade heartbeat sent: state=%s signals=%d", state, signals)
except asyncio.CancelledError:
raise
except Exception:
logger.exception("ai_trade heartbeat 실패 — 다음 주기에 재시도")
await asyncio.sleep(INTERVAL)
finally:
await redis.aclose()

View File

@@ -3,9 +3,12 @@ from __future__ import annotations
import asyncio
import logging
from contextlib import asynccontextmanager
from datetime import datetime
from zoneinfo import ZoneInfo
from fastapi import FastAPI
from ai_trade import heartbeat as _hb
from ai_trade import state as state_mod
from ai_trade.chronos_predictor import ChronosPredictor
from ai_trade.config import get_settings
@@ -13,8 +16,11 @@ from ai_trade.kis_client import KISClient
from ai_trade.kis_websocket import KISWebSocket
from ai_trade.pull_worker import poll_loop, make_asking_price_callback
from ai_trade.rate_limit import SignalDedup
from ai_trade.scheduler import _is_polling_window, _is_market_day
from ai_trade.stock_client import StockClient
_KST = ZoneInfo("Asia/Seoul")
logger = logging.getLogger(__name__)
@@ -23,6 +29,7 @@ class AppContext:
dedup: SignalDedup | None = None
shutdown: asyncio.Event | None = None
poll_task: asyncio.Task | None = None
hb_task: asyncio.Task | None = None
kis_client: KISClient | None = None
kis_ws: KISWebSocket | None = None
chronos: ChronosPredictor | None = None
@@ -87,9 +94,27 @@ async def lifespan(app: FastAPI):
)
)
def _trader_state() -> tuple[str, int]:
"""scheduler의 실제 폴링 윈도우 판정으로 market_open/market_closed 결정."""
now = datetime.now(_KST)
is_open = _is_market_day(now) and _is_polling_window(now)
state_str = "market_open" if is_open else "market_closed"
signals = len(state_mod.state.signals)
return state_str, signals
_ctx.hb_task = asyncio.create_task(_hb.heartbeat_loop(_trader_state))
yield
# Shutdown
# Shutdown heartbeat task
if _ctx.hb_task is not None:
_ctx.hb_task.cancel()
try:
await _ctx.hb_task
except asyncio.CancelledError:
pass
# Shutdown poll task
if _ctx.shutdown is not None:
_ctx.shutdown.set()
if _ctx.poll_task is not None:

View File

@@ -0,0 +1,38 @@
"""Tests for ai_trade heartbeat payload builder."""
import json
import pytest
def test_trader_payload_market_open():
from ai_trade.heartbeat import build_trader_payload
p = json.loads(build_trader_payload("market_open", signals=2))
assert p["name"] == "ai_trade"
assert p["kind"] == "trader"
assert p["state"] == "market_open"
assert p["ts"].endswith("Z")
assert p["jobs_done"] == 2
def test_trader_payload_market_closed():
from ai_trade.heartbeat import build_trader_payload
p = json.loads(build_trader_payload("market_closed"))
assert p["name"] == "ai_trade"
assert p["kind"] == "trader"
assert p["state"] == "market_closed"
assert p["jobs_done"] == 0
assert p["jobs_failed"] == 0
assert p["last_job_at"] is None
def test_trader_payload_ts_format():
"""ts 필드가 ISO 8601 UTC 형식 (YYYY-MM-DDTHH:MM:SSZ)인지 확인."""
from ai_trade.heartbeat import build_trader_payload
import re
p = json.loads(build_trader_payload("market_open"))
assert re.match(r"\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}Z", p["ts"]), (
f"ts={p['ts']!r} does not match expected UTC format"
)

View File

@@ -7,6 +7,7 @@ pytest>=8.0
pytest-asyncio>=0.23
respx>=0.21
websockets>=12
redis>=5.0
# Phase 3b dependencies (Chronos-2 + ML)
transformers>=4.40
chronos-forecasting>=1.4

View File

@@ -0,0 +1,55 @@
"""분산 워커 heartbeat — worker:<name>:heartbeat SET (TTL). Global Constraints 계약 1."""
from __future__ import annotations
import asyncio, datetime as dt, json, logging, os
logger = logging.getLogger(__name__)
DEFAULT_INTERVAL = int(os.getenv("HEARTBEAT_INTERVAL", "15"))
DEFAULT_TTL = int(os.getenv("HEARTBEAT_TTL", "45"))
class WorkerStats:
"""worker_loop가 갱신, heartbeat_loop가 읽는 가변 카운터."""
def __init__(self):
self.busy = False
self.jobs_done = 0
self.jobs_failed = 0
self.last_job_at = None # ISO str | None
def utc_now_iso() -> str:
return dt.datetime.now(dt.timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ")
def build_payload(name: str, kind: str, state: str, stats: WorkerStats, extra: dict | None = None) -> str:
payload = {
"name": name, "kind": kind, "state": state, "ts": utc_now_iso(),
"last_job_at": stats.last_job_at,
"jobs_done": stats.jobs_done, "jobs_failed": stats.jobs_failed,
}
if extra:
payload.update(extra)
return json.dumps(payload)
async def render_state(redis, stats: WorkerStats, paused_key: str = "queue:paused") -> str:
if await redis.get(paused_key) == b"1":
return "paused"
return "busy" if stats.busy else "idle"
async def heartbeat_loop(redis, name, kind, stats, *, interval=DEFAULT_INTERVAL,
ttl=DEFAULT_TTL, paused_key="queue:paused", state_fn=None):
key = f"worker:{name}:heartbeat"
logger.info("heartbeat 시작 name=%s ttl=%ds", name, ttl)
while True:
try:
if state_fn is not None:
state, extra = await state_fn(redis, stats)
else:
state, extra = await render_state(redis, stats, paused_key), None
await redis.set(key, build_payload(name, kind, state, stats, extra), ex=ttl)
except asyncio.CancelledError:
raise
except Exception:
logger.exception("heartbeat 발신 실패 name=%s", name)
await asyncio.sleep(interval)

View File

@@ -0,0 +1,46 @@
"""Tests for _shared.heartbeat — Task A1."""
import json
import sys
from pathlib import Path
import pytest
# Make `_shared` importable (same pattern as test_reliable_queue.py)
sys.path.insert(0, str(Path(__file__).resolve().parent.parent.parent))
from _shared.heartbeat import WorkerStats, build_payload, render_state
def test_build_payload_has_contract_fields():
s = WorkerStats(); s.jobs_done = 3; s.last_job_at = "2026-06-29T00:00:00Z"
payload = json.loads(build_payload("image-render", "render", "idle", s))
assert payload["name"] == "image-render"
assert payload["kind"] == "render"
assert payload["state"] == "idle"
assert payload["jobs_done"] == 3
assert payload["last_job_at"] == "2026-06-29T00:00:00Z"
assert payload["ts"].endswith("Z")
def test_build_payload_merges_extra():
payload = json.loads(build_payload("task-watcher", "watcher", "free", WorkerStats(), extra={"mode": "free"}))
assert payload["mode"] == "free"
class _FakeRedis:
def __init__(self, paused): self._paused = paused
async def get(self, key): return b"1" if self._paused else None
@pytest.mark.asyncio
async def test_render_state_paused_overrides_busy():
s = WorkerStats(); s.busy = True
assert await render_state(_FakeRedis(paused=True), s) == "paused"
@pytest.mark.asyncio
async def test_render_state_busy_then_idle():
s = WorkerStats(); s.busy = True
assert await render_state(_FakeRedis(paused=False), s) == "busy"
s.busy = False
assert await render_state(_FakeRedis(paused=False), s) == "idle"

View File

@@ -14,7 +14,7 @@ services:
- REDIS_URL=${REDIS_URL:-redis://192.168.45.54:6379}
- NAS_BASE_URL=${NAS_BASE_URL:-http://192.168.45.54:18700}
- INTERNAL_API_KEY=${INTERNAL_API_KEY:-}
- INSTA_MEDIA_ROOT=${INSTA_MEDIA_ROOT:-/mnt/nas/webpage/data/insta}
- INSTA_MEDIA_ROOT=${INSTA_MEDIA_ROOT:-/mnt/nas/webpage/data/insta/insta_cards}
- INSTA_MEDIA_URL_PREFIX=${INSTA_MEDIA_URL_PREFIX:-/media/insta}
- CARD_TEMPLATE_DIR=/app/templates
volumes:
@@ -82,7 +82,8 @@ services:
task-watcher:
build:
context: ./task-watcher
context: .
dockerfile: task-watcher/Dockerfile
container_name: task-watcher
restart: unless-stopped
ports:
@@ -126,3 +127,52 @@ services:
interval: 60s
timeout: 5s
retries: 3
trade-monitor:
build:
context: .
dockerfile: trade-monitor/Dockerfile
container_name: trade-monitor
restart: unless-stopped
ports:
- "18715:8000"
environment:
- TZ=Asia/Seoul
- REDIS_URL=${REDIS_URL:-redis://192.168.45.54:6379}
- NAS_BASE_URL=${NAS_BASE_URL:-http://192.168.45.54:18500}
- WEBAI_API_KEY=${WEBAI_API_KEY:-}
- TM_KIS_APP_KEY=${TM_KIS_APP_KEY:-}
- TM_KIS_APP_SECRET=${TM_KIS_APP_SECRET:-}
- TM_KIS_ACCOUNT=${TM_KIS_ACCOUNT:-}
- TM_KIS_IS_VIRTUAL=${TM_KIS_IS_VIRTUAL:-0}
- TM_LOOP_INTERVAL=${TM_LOOP_INTERVAL:-60}
- TM_CLIMAX_VOL_MULT=${TM_CLIMAX_VOL_MULT:-3.0}
healthcheck:
test: ["CMD", "python", "-c", "import urllib.request; urllib.request.urlopen('http://localhost:8000/health')"]
interval: 60s
timeout: 5s
retries: 3
naver-fetch:
build:
context: .
dockerfile: naver-fetch/Dockerfile
container_name: naver-fetch
restart: unless-stopped
ports:
- "18716:8000"
environment:
- TZ=Asia/Seoul
- REDIS_URL=${REDIS_URL:-redis://192.168.45.54:6379}
- NAS_BASE_URL=${NAVER_NAS_BASE_URL:-http://192.168.45.54:18800}
- INTERNAL_API_KEY=${INTERNAL_API_KEY:-}
- FETCH_INTERVAL_HOURS=${FETCH_INTERVAL_HOURS:-2.5}
- DAYTIME_START=${DAYTIME_START:-08:00}
- DAYTIME_END=${DAYTIME_END:-20:00}
- NAVER_REAL_ESTATE_TYPE=${NAVER_REAL_ESTATE_TYPE:-APT:ABYG:JGC:PRE}
- NAVER_HEADLESS=${NAVER_HEADLESS:-1}
healthcheck:
test: ["CMD", "python", "-c", "import urllib.request; urllib.request.urlopen('http://localhost:8000/health')"]
interval: 60s
timeout: 5s
retries: 3

View File

@@ -3,11 +3,14 @@ from __future__ import annotations
import asyncio
import logging
import os
from contextlib import asynccontextmanager
import redis.asyncio as aioredis
from fastapi import FastAPI
import worker
from _shared.heartbeat import heartbeat_loop
logging.basicConfig(level=logging.INFO, format="%(asctime)s %(name)s %(levelname)s %(message)s")
logger = logging.getLogger(__name__)
@@ -16,15 +19,19 @@ logger = logging.getLogger(__name__)
@asynccontextmanager
async def lifespan(app: FastAPI):
worker_task = asyncio.create_task(worker.worker_loop())
hb_redis = aioredis.from_url(os.getenv("REDIS_URL", "redis://192.168.45.54:6379"), decode_responses=False)
hb_task = asyncio.create_task(heartbeat_loop(hb_redis, "image-render", "render", worker.stats))
logger.info("image-render lifespan 시작")
try:
yield
finally:
worker_task.cancel()
for t in (worker_task, hb_task):
t.cancel()
try:
await worker_task
await t
except asyncio.CancelledError:
pass
await hb_redis.aclose()
logger.info("image-render lifespan 종료")

View File

@@ -67,3 +67,25 @@ async def test_poll_once_returns_false_on_timeout(monkeypatch):
assert handled is False
fake_queue.ack.assert_not_awaited()
fake_queue.fail.assert_not_awaited()
# ----- heartbeat stats 카운터 -----
class _OneJobQueue:
def __init__(self): self.acked = False
async def dequeue(self, timeout=5):
if self.acked: return None
return ({"job_type": "flux_generation", "task_id": "t1", "params": {}}, b"raw")
async def ack(self, raw): self.acked = True
async def fail(self, raw, payload): pass
@pytest.mark.asyncio
async def test_poll_once_increments_jobs_done(monkeypatch):
worker.stats.jobs_done = 0
monkeypatch.setattr(worker, "run_flux_generation", lambda task_id, params: None)
handled = await worker.poll_once(_OneJobQueue())
assert handled is True
assert worker.stats.jobs_done == 1
assert worker.stats.busy is False
assert worker.stats.last_job_at is not None

View File

@@ -18,6 +18,7 @@ from providers.gpt_image import run_gpt_image_generation
from providers.nano_banana import run_nano_banana_generation
from providers.flux import run_flux_generation
from _shared.reliable_queue import ReliableQueue
from _shared.heartbeat import WorkerStats, utc_now_iso
logger = logging.getLogger(__name__)
@@ -25,6 +26,8 @@ REDIS_URL = os.getenv("REDIS_URL", "redis://192.168.45.54:6379")
QUEUE_KEY = "queue:image-render"
PAUSED_KEY = "queue:paused"
stats = WorkerStats()
# string names so `unittest.mock.patch` / `monkeypatch.setattr` on `worker.<name>`
# is correctly intercepted by getattr(sys.modules[__name__], ...)
_DISPATCH_TABLE = {
@@ -59,14 +62,21 @@ async def poll_once(queue: ReliableQueue) -> bool:
if result is None:
return False
payload, raw = result
stats.busy = True
try:
await asyncio.to_thread(_dispatch, payload)
except Exception:
logger.exception("dispatch unhandled exception task_id=%s",
payload.get("task_id"))
await queue.fail(raw, payload)
stats.jobs_failed += 1
stats.last_job_at = utc_now_iso()
stats.busy = False
return True
await queue.ack(raw)
stats.jobs_done += 1
stats.last_job_at = utc_now_iso()
stats.busy = False
return True

View File

@@ -7,8 +7,9 @@ REDIS_URL=redis://192.168.45.54:6379
NAS_BASE_URL=http://192.168.45.54:18700
INTERNAL_API_KEY=__copy_from_nas_dotenv__
# NAS SMB mount 안의 미디어 디렉토리 (/mnt/nas/webpage/data/insta/)
INSTA_MEDIA_ROOT=/mnt/nas/webpage/data/insta
# NAS SMB mount 안의 미디어 디렉토리.
# ⚠️ nginx가 /media/insta를 data/insta/insta_cards/로 서빙하므로 반드시 insta_cards까지 포함.
INSTA_MEDIA_ROOT=/mnt/nas/webpage/data/insta/insta_cards
# nginx 서빙 prefix (NAS webhook payload에 보낼 result_path 만들 때)
INSTA_MEDIA_URL_PREFIX=/media/insta

View File

@@ -151,8 +151,11 @@ async def _render_slate_locked(slate: dict, slate_id: int, template: str) -> Lis
html_path = f.name
try:
await page.goto(f"file://{html_path}", wait_until="networkidle")
await page.evaluate("document.fonts.ready") # 웹폰트 로딩 완료까지 대기
out_path = os.path.join(out_dir, f"{spec['page_no']:02d}.png")
await page.screenshot(path=out_path, full_page=False, omit_background=False)
if os.path.getsize(out_path) < 1000: # 빈/깨진 PNG 방어
raise RuntimeError(f"rendered PNG too small: {out_path}")
paths.append(out_path)
finally:
try:

View File

@@ -3,12 +3,15 @@ from __future__ import annotations
import asyncio
import logging
import os
from contextlib import asynccontextmanager
import redis.asyncio as aioredis
from fastapi import FastAPI
import card_renderer
import worker
from _shared.heartbeat import heartbeat_loop
logging.basicConfig(level=logging.INFO, format="%(asctime)s %(name)s %(levelname)s %(message)s")
logger = logging.getLogger(__name__)
@@ -20,15 +23,19 @@ async def lifespan(app: FastAPI):
await card_renderer.init_browser()
# 큐 워커 백그라운드 시작
worker_task = asyncio.create_task(worker.worker_loop())
hb_redis = aioredis.from_url(os.getenv("REDIS_URL", "redis://192.168.45.54:6379"), decode_responses=False)
hb_task = asyncio.create_task(heartbeat_loop(hb_redis, "insta-render", "render", worker.stats))
logger.info("insta-render lifespan 시작")
try:
yield
finally:
worker_task.cancel()
for t in (worker_task, hb_task):
t.cancel()
try:
await worker_task
await t
except asyncio.CancelledError:
pass
await hb_redis.aclose()
await card_renderer.shutdown_browser()
logger.info("insta-render lifespan 종료")

View File

@@ -3,52 +3,85 @@
<head>
<meta charset="UTF-8">
<style>
@import url('https://fonts.googleapis.com/css2?family=Noto+Sans+KR:wght@400;700;900&display=swap');
@import url('https://cdn.jsdelivr.net/gh/orioncactus/pretendard@v1.3.9/dist/web/static/pretendard.css');
* { margin: 0; padding: 0; box-sizing: border-box; }
html, body {
width: 1080px; height: 1350px;
font-family: 'Noto Sans KR', sans-serif;
html, body { width: 1080px; height: 1350px; }
body {
font-family: 'Pretendard', 'Noto Sans KR', sans-serif;
background: #F7F7FA; color: #14171A;
-webkit-font-smoothing: antialiased;
}
.card {
width: 1080px; height: 1350px;
padding: 80px 72px;
display: flex; flex-direction: column; justify-content: space-between;
background: linear-gradient(180deg, #FFFFFF 0%, #F7F7FA 100%);
border-top: 16px solid {{ accent_color }};
position: relative; width: 1080px; height: 1350px; overflow: hidden;
padding: 96px 84px 72px;
display: flex; flex-direction: column;
background: #FFFFFF;
}
.accent-bar { position: absolute; top: 0; left: 0; width: 100%; height: 14px; background: {{ accent_color | safe }}; }
.badge {
display: inline-block; padding: 8px 20px; border-radius: 999px;
background: {{ accent_color }}; color: #fff;
font-size: 28px; font-weight: 700; letter-spacing: -0.02em;
align-self: flex-start; padding: 10px 24px; border-radius: 999px;
background: {{ accent_color | safe }}; color: #fff;
font-size: 30px; font-weight: 700; letter-spacing: -0.02em;
}
.idx { font-size: 120px; font-weight: 800; line-height: 1; color: {{ accent_color | safe }}; letter-spacing: -0.04em; }
.content { flex: 1; display: flex; flex-direction: column; justify-content: center; gap: 36px; }
.headline {
font-size: {{ 96 if page_type == 'cover' else 72 }}px;
font-weight: 900; line-height: 1.15; letter-spacing: -0.04em;
margin-top: 32px;
font-weight: 800; line-height: 1.18; letter-spacing: -0.04em; color: #14171A;
display: -webkit-box; -webkit-box-orient: vertical; overflow: hidden;
}
.body {
font-size: 40px; font-weight: 400; line-height: 1.55;
margin-top: 40px; color: #2A2F35;
.cover .headline { font-size: 104px; -webkit-line-clamp: 4; }
.body-page .headline { font-size: 76px; -webkit-line-clamp: 3; }
.cta .headline { font-size: 88px; -webkit-line-clamp: 3; }
.sub {
font-size: 42px; font-weight: 400; line-height: 1.5; color: #3A4047;
display: -webkit-box; -webkit-box-orient: vertical; overflow: hidden; -webkit-line-clamp: 8;
white-space: pre-wrap;
}
.cover .sub { -webkit-line-clamp: 5; }
.footer {
display: flex; justify-content: space-between; align-items: center;
font-size: 28px; color: #6B7280; font-weight: 500;
font-size: 28px; color: #8A9099; font-weight: 600; margin-top: 40px;
}
.cta { font-weight: 700; color: {{ accent_color }}; }
.cta-pill {
align-self: flex-start; margin-top: 8px; padding: 18px 40px; border-radius: 16px;
background: {{ accent_color | safe }}; color: #fff; font-size: 40px; font-weight: 700;
}
.progress { display: flex; gap: 10px; }
.progress i { width: 14px; height: 14px; border-radius: 50%; background: #D8DCE0; display: inline-block; }
.progress i.on { background: {{ accent_color | safe }}; }
</style>
</head>
<body>
<div class="card">
<div>
<span class="badge">{{ page_type|upper }}</span>
<div class="card {{ 'cover' if page_type=='cover' else ('cta' if page_type=='cta' else 'body-page') }}">
<div class="accent-bar"></div>
{% if page_type == 'cover' %}
<span class="badge">{{ category_label|default('') or '오늘의 이슈' }}</span>
<div class="content">
<h1 class="headline">{{ headline }}</h1>
<p class="body">{{ body }}</p>
<p class="sub">{{ body }}</p>
</div>
{% elif page_type == 'cta' %}
<div class="content">
<h1 class="headline">{{ headline }}</h1>
<p class="sub">{{ body }}</p>
{% if cta %}<div class="cta-pill">{{ cta }}</div>{% endif %}
</div>
{% else %}
<span class="idx">{{ '%02d'|format(page_no - 1) }}</span>
<div class="content">
<h1 class="headline">{{ headline }}</h1>
<p class="sub">{{ body }}</p>
</div>
{% endif %}
<div class="footer">
{% if page_type == 'cover' or page_type == 'cta' %}
<span>{{ brand_handle|default('') }}</span><span>{{ page_no }} / {{ total_pages }}</span>
{% else %}
<div class="progress">{% for n in range(2, total_pages) %}<i class="{{ 'on' if n <= page_no }}"></i>{% endfor %}</div>
<span>{{ page_no }} / {{ total_pages }}</span>
{% if cta %}<span class="cta">{{ cta }}</span>{% endif %}
{% endif %}
</div>
</div>
</body>

View File

@@ -1,10 +1,13 @@
"""worker.py — Redis BLPOP + webhook 단위 테스트."""
import json
import os
from pathlib import Path
import pytest
import httpx
from unittest.mock import AsyncMock, patch
import worker
from card_renderer import render_slate, init_browser, shutdown_browser
@pytest.fixture
@@ -179,6 +182,28 @@ async def test_poll_once_calls_fail_on_exception(monkeypatch):
fake_queue.ack.assert_not_awaited()
@pytest.mark.asyncio
async def test_render_produces_nonempty_1080x1350(tmp_path, monkeypatch):
"""Phase 2 — fonts.ready 대기 + PNG 비어있음 검증: 10장 모두 > 1000 bytes."""
import card_renderer as _cr
templates_dir = str(Path(__file__).resolve().parent.parent / "templates")
monkeypatch.setattr(_cr, "CARD_TEMPLATE_DIR", templates_dir)
monkeypatch.setattr(_cr, "INSTA_MEDIA_ROOT", str(tmp_path))
await init_browser()
try:
slate = {
"cover_copy": {"headline": "헤드라인", "body": "서브", "accent_color": "#0F62FE"},
"body_copies": [{"headline": f"포인트{i}", "body": "본문"} for i in range(8)],
"cta_copy": {"headline": "요약", "body": "마무리", "cta": "팔로우"},
}
paths = await render_slate(slate, slate_id=99999)
assert len(paths) == 10
for p in paths:
assert os.path.getsize(p) > 1000 # 비어있지 않음
finally:
await shutdown_browser()
@pytest.mark.asyncio
async def test_poll_once_returns_false_on_timeout(monkeypatch):
"""F6 — dequeue가 None 반환(타임아웃)이면 False 리턴, ack/fail 안 부름."""
@@ -197,3 +222,35 @@ async def test_poll_once_returns_false_on_timeout(monkeypatch):
process_mock.assert_not_awaited()
fake_queue.ack.assert_not_awaited()
fake_queue.fail.assert_not_awaited()
def test_make_queue_redis_socket_timeout_exceeds_block():
"""BLMOVE(블록 5s) dequeue가 read-timeout 경계 경합으로 깨지지 않도록
socket_timeout이 블록보다 충분히 커야 한다 (회귀 가드)."""
c = worker.make_queue_redis()
st = c.connection_pool.connection_kwargs.get("socket_timeout")
assert st is not None and st > 5 # blmove 블록(5s)보다 커야 안정
# ----- heartbeat stats 카운터 -----
class _OneJobQueueInsta:
def __init__(self): self.acked = False
async def dequeue(self, timeout=5):
if self.acked: return None
return ({"task_id": "t1", "params": {"slate_id": 1, "theme": "default"}}, b"raw")
async def ack(self, raw): self.acked = True
async def fail(self, raw, payload): pass
@pytest.mark.asyncio
async def test_poll_once_increments_jobs_done(monkeypatch):
worker.stats.jobs_done = 0
async def fake_process(client, payload): pass
monkeypatch.setattr(worker, "_process_one", fake_process)
async with httpx.AsyncClient() as client:
handled = await worker.poll_once(_OneJobQueueInsta(), client)
assert handled is True
assert worker.stats.jobs_done == 1
assert worker.stats.busy is False
assert worker.stats.last_job_at is not None

View File

@@ -14,9 +14,11 @@ import redis.asyncio as aioredis
from card_renderer import render_slate
from _shared.reliable_queue import ReliableQueue
from _shared.heartbeat import WorkerStats, utc_now_iso
logger = logging.getLogger(__name__)
stats = WorkerStats()
REDIS_URL = os.getenv("REDIS_URL", "redis://192.168.45.54:6379")
NAS_BASE_URL = os.getenv("NAS_BASE_URL", "http://192.168.45.54:18700")
@@ -89,18 +91,40 @@ async def poll_once(queue: ReliableQueue, client: httpx.AsyncClient) -> bool:
if result is None:
return False
payload, raw = result
stats.busy = True
try:
await _process_one(client, payload)
except Exception:
await queue.fail(raw, payload)
stats.jobs_failed += 1
stats.last_job_at = utc_now_iso()
stats.busy = False
return True
await queue.ack(raw)
stats.jobs_done += 1
stats.last_job_at = utc_now_iso()
stats.busy = False
return True
# 블로킹 dequeue는 BLMOVE(블록 5s)를 쓴다. redis-py 블로킹 read에서 socket_timeout이
# 블록(5s) 이하이거나 None이면 read-timeout이 블록 경계와 경합해 간헐적으로
# "Timeout reading"이 터져 잡을 못 꺼낸다(슬레이트 draft 정지). 실험상 socket_timeout이
# 블록보다 충분히 크면(10/30) 항상 안정. → 블록보다 넉넉히 큰 값을 명시한다.
QUEUE_SOCKET_TIMEOUT = 30 # > dequeue blmove 블록(5s)
def make_queue_redis():
"""블로킹 dequeue(BLMOVE)용 redis 클라이언트. socket_timeout > 블록(5s) 보장."""
return aioredis.from_url(
REDIS_URL, decode_responses=False,
socket_timeout=QUEUE_SOCKET_TIMEOUT, socket_keepalive=True,
)
async def worker_loop():
"""무한 루프 — paused 체크 → ReliableQueue.dequeue → process_one → ack/fail."""
redis = aioredis.from_url(REDIS_URL, decode_responses=False)
redis = make_queue_redis()
queue = ReliableQueue(redis, queue_key=QUEUE_KEY)
async with httpx.AsyncClient() as client:
logger.info("insta-render worker started worker_id=%s queue=%s",

View File

@@ -7,12 +7,15 @@ from __future__ import annotations
import asyncio
import logging
import os
from contextlib import asynccontextmanager
import redis.asyncio as aioredis
from fastapi import FastAPI, HTTPException
from pydantic import BaseModel
import worker
from _shared.heartbeat import heartbeat_loop
from providers.sync_ops import (
generate_lyrics, get_credits,
get_timestamped_lyrics, generate_style_boost,
@@ -25,15 +28,19 @@ logger = logging.getLogger(__name__)
@asynccontextmanager
async def lifespan(app: FastAPI):
worker_task = asyncio.create_task(worker.worker_loop())
hb_redis = aioredis.from_url(os.getenv("REDIS_URL", "redis://192.168.45.54:6379"), decode_responses=False)
hb_task = asyncio.create_task(heartbeat_loop(hb_redis, "music-render", "render", worker.stats))
logger.info("music-render lifespan 시작")
try:
yield
finally:
worker_task.cancel()
for t in (worker_task, hb_task):
t.cancel()
try:
await worker_task
await t
except asyncio.CancelledError:
pass
await hb_redis.aclose()
logger.info("music-render lifespan 종료")

View File

@@ -167,3 +167,25 @@ async def test_poll_once_returns_false_on_timeout(monkeypatch):
dispatch_mock.assert_not_called()
fake_queue.ack.assert_not_awaited()
fake_queue.fail.assert_not_awaited()
# ----- heartbeat stats 카운터 -----
class _OneJobQueue:
def __init__(self): self.acked = False
async def dequeue(self, timeout=5):
if self.acked: return None
return ({"job_type": "suno_generation", "task_id": "t1", "params": {}}, b"raw")
async def ack(self, raw): self.acked = True
async def fail(self, raw, payload): pass
@pytest.mark.asyncio
async def test_poll_once_increments_jobs_done(monkeypatch):
worker.stats.jobs_done = 0
monkeypatch.setattr(worker, "run_suno_generation", lambda task_id, params: None)
handled = await worker.poll_once(_OneJobQueue())
assert handled is True
assert worker.stats.jobs_done == 1
assert worker.stats.busy is False
assert worker.stats.last_job_at is not None

View File

@@ -21,6 +21,7 @@ from providers.suno import (
)
from providers.local import run_local_generation
from _shared.reliable_queue import ReliableQueue
from _shared.heartbeat import WorkerStats, utc_now_iso
logger = logging.getLogger(__name__)
@@ -28,6 +29,8 @@ REDIS_URL = os.getenv("REDIS_URL", "redis://192.168.45.54:6379")
QUEUE_KEY = "queue:music-render"
PAUSED_KEY = "queue:paused"
stats = WorkerStats()
# Maps job_type → module-level function name (string).
# _dispatch resolves the name via globals() at call time so unittest.mock.patch
# on "worker.<name>" is correctly intercepted.
@@ -74,6 +77,7 @@ async def poll_once(queue: ReliableQueue) -> bool:
if result is None:
return False
payload, raw = result
stats.busy = True
try:
# sync provider 함수 — thread로 실행해서 이벤트 루프 블로킹 방지
await asyncio.to_thread(_dispatch, payload)
@@ -81,8 +85,14 @@ async def poll_once(queue: ReliableQueue) -> bool:
logger.exception("dispatch unhandled exception task_id=%s",
payload.get("task_id"))
await queue.fail(raw, payload)
stats.jobs_failed += 1
stats.last_job_at = utc_now_iso()
stats.busy = False
return True
await queue.ack(raw)
stats.jobs_done += 1
stats.last_job_at = utc_now_iso()
stats.busy = False
return True

View File

@@ -0,0 +1,17 @@
# naver-fetch — realestate 매물 fetch 워커
REDIS_URL=redis://192.168.45.54:6379
# realestate-lab 직결 (targets/listings-ingest). compose가 NAVER_NAS_BASE_URL로 주입
NAVER_NAS_BASE_URL=http://192.168.45.54:18800
# render 워커 공유키 (X-Internal-Key)
INTERNAL_API_KEY=
# 스케줄 (KST)
FETCH_INTERVAL_HOURS=2.5
DAYTIME_START=08:00
DAYTIME_END=20:00
# 네이버
NAVER_REAL_ESTATE_TYPE=APT:ABYG:JGC:PRE
NAVER_HEADLESS=1
# NAVER_TOKEN_URL= # 기본값(new.land.naver.com houses) 사용

View File

@@ -0,0 +1,152 @@
# naver-fetch 워커 — 구현 설계
> **2026-07-10 · web-ai 소유.** realestate 매물 알림 파이프라인의 가정-IP fetch 워커.
> **권위 계약(원본 스펙)**: web-backend repo `realestate-lab/docs/superpowers/specs/2026-07-09-naver-listing-worker-design.md` §4.
> 이 문서는 그 계약을 Windows Docker 워커로 구현하기 위한 **구현 설계**.
---
## 1. 역할 · 경계
네이버가 NAS(데이터센터) IP를 429/ReadTimeout으로 막기 때문에, **가정 IP(LAN 192.168.45.x)에서 네이버 매물을 fetch해 NAS로 전달**하는 얇은 프록시 워커. trade-monitor·render 워커와 동일한 FastAPI + asyncio 관례. WSL2 Docker.
**책임**
1. 2~3h 주간 주기로 NAS `targets` 조회(감시 동 목록).
2. Playwright로 네이버 Bearer 토큰 획득 → httpx로 동별 매물 fetch(페이지네이션).
3. raw article을 그대로 NAS `listings-ingest`로 전달(**파싱·매칭·알림은 NAS**).
4. Redis heartbeat(`worker:naver-fetch:heartbeat` EX45, kind=fetcher).
5. 토큰/네트워크 오류를 사이클·동 단위로 격리.
**경계 밖(안 함)**: article 파싱, 매물 매칭, 텔레그램 발송(전부 NAS), dedup.
### Spike 검증 완료 (2026-07-10)
- Playwright 헤드리스 → `new.land.naver.com` 로드 → `/api/articles` XHR의 **Bearer JWT 토큰 인터셉트 성공**.
- 인터셉트 토큰+쿠키를 httpx로 재사용 → `GET /api/articles?cortarNo=1159010100&...` **HTTP 200, articleList 20건**(isMoreData 페이지네이션). 토큰 있으면 이 LAN에서 429 회피됨.
- raw article 키: `articleNo, articleName, tradeTypeName, dealOrWarrantPrc, areaName, buildingName, floorInfo` 등 → NAS로 그대로 전달.
---
## 2. 모듈 분해
| 파일 | 책임 | 인터페이스 |
|------|------|-----------|
| `main.py` | FastAPI + lifespan. `fetch_loop` + `heartbeat_loop` 스폰, `/health` | 부작용 |
| `fetch_worker.py` | `FetchState`, `run_cycle`, `fetch_loop`, `heartbeat_loop`(전용) | 부작용 |
| `nas_client.py` | `get_targets()` / `post_ingest(fetched_at, batches)` (X-Internal-Key + retry) | 부작용(HTTP) |
| `naver_client.py` | `NaverClient`: `acquire_token()`(Playwright) + `fetch_articles(cortar_no, page)`(httpx) | 부작용(브라우저/HTTP) |
| `config.py` | `Settings` — env 로드 | 순수 |
순수 로직이 적은 워커(얇은 프록시)라 파싱은 없음. 테스트는 HTTP mock(respx) + fetch_worker 조립 중심.
---
## 3. 데이터 흐름 (run_cycle)
```
state.state = "fetching"
await naver.acquire_token() # Playwright 1회 (사이클당)
tg = await nas.get_targets() # {dongs:[{dong,cortar_no}], deal_types, page_limit}
for d in tg.dongs: # 동 단위 try/except 격리 (fetch/ingest 각각)
articles = []
for page in 1..tg.page_limit:
resp = await naver.fetch_articles(d.cortar_no, page) # httpx GET
articles += resp.get("articleList", [])
if not resp.get("isMoreData"): break
# ⚠️ 동별로 즉시 ingest — 전체 batch를 한 POST로 보내면 NAS 동기 처리(매물 수 비례)가
# >180s 타임아웃. 동 단위(각 수십건)로 쪼개 각 POST가 timeout 내 완료 + 부분 실패 격리.
r = await nas.post_ingest(fetched_at=iso_utc_now, batches=[{"dong": d.dong, "articles": articles}])
state.listings_pushed += len(articles) # {received,new,matched}
state.last_fetch_at = epoch_now
state.state = "idle"
```
> ⚠️ **ingest 분할**: 스펙은 batches(복수)를 한 POST로 예시하지만, 첫 실행 대량(6동×~40건)이 NAS에서 >180s 걸려 타임아웃 → **동별 1 POST**로 분할(batches 리스트 계약 호환, 각 POST는 batches 1개). nas_client의 ingest는 timeout 180s·재시도 없음(재전송 시 NAS 중복 알림 방지). BE에 분할 통지 필요.
> ⚠️ **시간 포맷 2종**: ingest `fetched_at`은 **ISO UTC 문자열**(스펙 §4 ingest 스키마, 예 `2026-07-09T05:44:23Z`), heartbeat `ts`/`last_fetch_at`은 **epoch 정수**(스펙 §4.4). 혼동 주의.
`heartbeat_loop`(별도 15초)이 `FetchState`를 읽어 §4.4 페이로드 발신.
---
## 4. naver_client (Playwright 토큰 + httpx fetch)
- `async acquire_token()`: `async_playwright`로 chromium 헤드리스 기동 → `new.land.naver.com/houses?ms=...`(설정 `NAVER_TOKEN_URL`) 로드 → `page.on("request")`로 첫 `Authorization: Bearer ...` 헤더 캡처 → `self._token`, `self._cookies`(context.cookies) 저장 → 브라우저 종료. 실패 시 예외(사이클 skip).
- **토큰 수명 정책**: 사이클마다 fresh 획득(2~3h당 1회). JWT 만료 무관, 캐시/갱신 로직 없음(단순).
- `async fetch_articles(cortar_no, page) -> dict`: httpx GET `https://new.land.naver.com/api/articles?cortarNo=<cortar_no>&order=rank&realEstateType=<NAVER_REAL_ESTATE_TYPE>&tradeType=&...&page=<page>` with headers `{authorization: self._token, user-agent, referer: https://new.land.naver.com/, accept, cookie: self._cookies}`. 200 아니면 예외. raw JSON dict 반환.
- `async close()`: httpx client 정리.
**얇은 프록시**: tradeType 비움(전체 유형) + realEstateType 설정값(기본 `APT:ABYG:JGC:PRE`). deal_types 필터는 NAS 담당(targets의 deal_types는 워커가 그대로 두고 NAS가 매칭 시 사용). 추후 네이버 쿼리 최적화 여지.
---
## 5. heartbeat (§4.4 — 전용 스키마)
`worker:naver-fetch:heartbeat` EX45. **trade-monitor의 `_shared.build_payload`(ISO ts + jobs_done/jobs_failed)와 스키마가 달라 전용 emit 작성**:
```json
{"name":"naver-fetch","kind":"fetcher","state":"idle|fetching",
"ts":<epoch int>,"last_fetch_at":<epoch int|null>,"listings_pushed":<int>,"errors":<int>}
```
- 15초 독립 태스크(주기 fetch가 2~3h라 TTL45 유지 위해 필수). `FetchState`를 읽어 발신.
- BE가 node_monitor.WORKER_REGISTRY에 `naver-fetch`(kind=fetcher)로 등재 완료 → 발신 시 `/nodes`·`/infra`에서 alive.
---
## 6. 스케줄 (fetch_loop)
- env `FETCH_INTERVAL_HOURS`(기본 2.5), `DAYTIME_START`/`DAYTIME_END`(기본 08:00/20:00 KST).
- 루프: ~5분마다 판정 — **주간창 안 && (now - last_run) ≥ interval** 이면 `run_cycle` 실행 후 `last_run=now`. 창 밖이면 idle(state=idle, fetch 안 함).
- 최상위 try/except로 루프 불사(어떤 예외도 죽이지 않음).
---
## 7. 설정 (env)
| env | 기본값 | 용도 |
|-----|--------|------|
| `NAS_BASE_URL` (compose: `NAVER_NAS_BASE_URL`) | `http://192.168.45.54:18800` | **realestate-lab 직결** targets/ingest (stock :18500 아님) |
| `INTERNAL_API_KEY` | (필수) | `X-Internal-Key` (render 워커 공유키) |
| `REDIS_URL` | `redis://192.168.45.54:6379` | heartbeat |
| `FETCH_INTERVAL_HOURS` | `2.5` | fetch 주기 |
| `DAYTIME_START` / `DAYTIME_END` | `08:00` / `20:00` | 주간창(KST) |
| `NAVER_REAL_ESTATE_TYPE` | `APT:ABYG:JGC:PRE` | 네이버 realEstateType |
| `NAVER_TOKEN_URL` | (기본 상수) | Playwright 토큰 획득용 네이버 URL |
| `NAVER_HEADLESS` | `1` | Playwright headless |
---
## 8. 에러 처리
- **토큰 획득 실패**: 사이클 skip, `errors++`, state=idle, 다음 주기 재시도.
- **targets 실패**: 사이클 skip, `errors++`.
- **동별 fetch 실패**: 해당 동 skip(warning), 나머지 동 계속.
- **ingest 실패**: 로그 error, `errors++`(다음 주기 재fetch).
- 루프 최상위 try/except.
---
## 9. 테스트 (pytest, 시스템 Python)
| 파일 | 검증 |
|------|------|
| `test_config.py` | env 기본/override |
| `test_nas_client.py` | respx — targets 파싱, ingest 페이로드/응답, X-Internal-Key, retry |
| `test_naver_client.py` | respx — fetch_articles URL·헤더(authorization/cookie)·articleList 파싱·비200 예외. (Playwright acquire_token은 실브라우저 통합이라 단위 mock: 토큰 주입 후 fetch 경로만) |
| `test_fetch_worker.py` | mock: 주간창 게이트(창 밖 skip), run_cycle 동별 조립·batches, 동 fetch 실패 격리, listings_pushed/last_fetch_at 갱신, heartbeat 필드(epoch ts/kind=fetcher) |
---
## 10. 배포
- `Dockerfile`: Playwright 베이스(`mcr.microsoft.com/playwright/python` 또는 insta-render 관례) + chromium, `COPY _shared /app/_shared`(빌드 컨텍스트 `.`), `COPY naver-fetch/. /app/`, `PYTHONPATH=/app`, uvicorn `:8000`.
- `services/docker-compose.yml`: `naver-fetch` 서비스, 포트 **18716**, `TZ=Asia/Seoul`, INTERNAL_API_KEY/REDIS/NAS env, healthcheck `/health`.
- 엔드포인트는 nginx `/api/internal/realestate/`(IP 화이트리스트, render 워커 IP 재사용).
---
## 11. 미해결 플래그 / 후속
1. **`INTERNAL_API_KEY` vs `REALESTATE_INTERNAL_KEY`** — 스펙 §config는 후자, BE 메시지는 전자(render 공유키). **BE 메시지 우선(INTERNAL_API_KEY)**, 배포 전 co-gahusb로 1줄 확인.
2. **네이버 헤드리스 탐지** — 현재 spike는 통과. 차단 시 stealth 헤더/headed 필요할 수 있음(첫 운영 모니터링).
3. **`isMoreData`/페이지 상한** — page_limit로 제한, 네이버 rate limit 재유입 시 동 간 지연 고려(첫 운영).
4. **deal_types 미사용** — 워커는 전체 유형 fetch, NAS가 필터. 필요 시 tradeType 매핑으로 최적화.
5. **토큰 재사용 실패(만료) 대비** — 사이클 중 fetch가 401 반환 시 acquire_token 1회 재시도(구현 시 반영 검토).

View File

@@ -0,0 +1,25 @@
FROM python:3.12-slim-bookworm
ENV PYTHONUNBUFFERED=1
WORKDIR /app
# Chromium 런타임 deps (Debian 12 / bookworm) — insta-render 관례
RUN apt-get update && apt-get install -y --no-install-recommends \
ca-certificates tzdata fonts-noto-cjk \
libnss3 libnspr4 libdbus-1-3 libatk1.0-0 libatk-bridge2.0-0 \
libcups2 libdrm2 libxkbcommon0 libxcomposite1 libxdamage1 \
libxfixes3 libxrandr2 libgbm1 libxshmfence1 libpango-1.0-0 \
libcairo2 libasound2 libatspi2.0-0 \
&& rm -rf /var/lib/apt/lists/*
COPY naver-fetch/requirements.txt /app/
RUN pip install --no-cache-dir --timeout 600 --retries 5 -r requirements.txt
RUN playwright install chromium
# 공통 모듈 (services/_shared) — 빌드 컨텍스트 . 필요 (task-watcher 함정)
COPY _shared /app/_shared
COPY naver-fetch/. /app/
ENV PYTHONPATH=/app
EXPOSE 8000
CMD ["python", "-m", "uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000", "--workers", "1"]

View File

@@ -0,0 +1,979 @@
# naver-fetch 워커 구현 계획
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** 네이버 매물을 가정 IP에서 fetch해 NAS로 전달하는 `naver-fetch` 워커를 구현한다 — Playwright로 Bearer 토큰 획득 → httpx로 동별 article fetch → NAS `listings-ingest`로 raw 전달 + heartbeat.
**Architecture:** FastAPI + asyncio 워커(trade-monitor·insta-render 관례). `naver_client`가 Playwright(토큰)+httpx(fetch)를 캡슐화, `fetch_worker`가 2~3h 주간 스케줄로 오케스트레이션. 얇은 프록시(파싱은 NAS). heartbeat는 fetcher 전용 스키마(epoch ts).
**Tech Stack:** Python 3.12, FastAPI, playwright==1.48.0(async, chromium), httpx(async), redis.asyncio, pytest + pytest-asyncio + respx. WSL2 Docker.
**설계 원본:** `services/naver-fetch/DESIGN.md` · 권위 계약: web-backend `realestate-lab/docs/superpowers/specs/2026-07-09-naver-listing-worker-design.md`.
## Global Constraints
- Python `3.12`, `playwright==1.48.0` (async API, chromium).
- **얇은 프록시** — raw article 그대로 전달, 파싱/매칭/알림/dedup은 NAS.
- 인증: `.env` `INTERNAL_API_KEY``X-Internal-Key` 헤더(render 워커 공유키). (스펙 §config의 `REALESTATE_INTERNAL_KEY`와 상충 — BE 메시지 우선, 배포 전 확인.)
- heartbeat 키 `worker:naver-fetch:heartbeat`, TTL **EX45**, kind `fetcher`, **epoch ts**: `{name,kind,state:idle|fetching,ts,last_fetch_at,listings_pushed,errors}`.
- ingest `fetched_at`**ISO UTC 문자열**(`YYYY-MM-DDTHH:MM:SSZ`).
- 스케줄: 주간창(기본 08:00~20:00 KST) 안에서 `FETCH_INTERVAL_HOURS`(기본 2.5) 간격.
- 테스트 실행: `C:\Users\jaeoh\AppData\Local\Programs\Python\Python312\python.exe -m pytest services/naver-fetch/tests -q` (web-ai 루트). 이하 `python` = 이 경로.
- 커밋: **web-ai repo에서만**. Docker 포트 **18716**.
- Spike 검증됨(2026-07-10): Playwright 토큰 인터셉트 + httpx `/api/articles` 200 + articleList.
---
## File Structure
| 파일 | 책임 |
|------|------|
| `services/naver-fetch/config.py` | `Settings` + `load_settings()` |
| `services/naver-fetch/nas_client.py` | `NASClient`: get_targets / post_ingest (X-Internal-Key) |
| `services/naver-fetch/naver_client.py` | `NaverClient`: acquire_token(Playwright) + fetch_articles(httpx) |
| `services/naver-fetch/fetch_worker.py` | `FetchState`, `in_daytime`, `run_cycle`, `fetch_loop`, `build_heartbeat`, `heartbeat_loop` |
| `services/naver-fetch/main.py` | FastAPI lifespan + `/health` |
| `services/naver-fetch/conftest.py` | services/ 루트 sys.path |
| `services/naver-fetch/tests/*` | 단위 테스트 |
| `services/naver-fetch/Dockerfile` | Playwright 베이스(insta-render 관례) + `_shared` |
| `services/naver-fetch/requirements.txt`, `.env.example`, `pytest.ini` | |
| `services/docker-compose.yml` (수정) | `naver-fetch` 서비스(18716) |
---
### Task 1: 스캐폴딩 + config
**Files:**
- Create: `services/naver-fetch/requirements.txt`, `conftest.py`, `pytest.ini`, `tests/__init__.py`, `config.py`
- Test: `services/naver-fetch/tests/test_config.py`
**Interfaces:**
- Produces: `Settings` dataclass fields `nas_base_url:str, internal_api_key:str, redis_url:str, fetch_interval_hours:float, daytime_start:str, daytime_end:str, naver_real_estate_type:str, naver_token_url:str, naver_headless:bool`. `load_settings() -> Settings`.
- [ ] **Step 1: 스캐폴딩**
`services/naver-fetch/requirements.txt`:
```
fastapi==0.115.6
uvicorn[standard]==0.34.0
playwright==1.48.0
redis>=5.0
httpx>=0.27
pytest>=8.0
pytest-asyncio>=0.24
respx>=0.21
```
`services/naver-fetch/conftest.py`:
```python
"""services/ 루트를 sys.path에 추가 — from _shared... import 가능하게."""
import sys
from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
```
`services/naver-fetch/pytest.ini`:
```
[pytest]
asyncio_mode = auto
```
`services/naver-fetch/tests/__init__.py`: (빈 파일)
- [ ] **Step 2: 실패 테스트**`services/naver-fetch/tests/test_config.py`
```python
"""Settings env 로드."""
from config import load_settings
def test_defaults(monkeypatch):
for k in ("NAS_BASE_URL", "INTERNAL_API_KEY", "REDIS_URL", "FETCH_INTERVAL_HOURS",
"DAYTIME_START", "DAYTIME_END", "NAVER_REAL_ESTATE_TYPE",
"NAVER_TOKEN_URL", "NAVER_HEADLESS"):
monkeypatch.delenv(k, raising=False)
s = load_settings()
assert s.nas_base_url == "http://192.168.45.54:18500"
assert s.fetch_interval_hours == 2.5
assert s.daytime_start == "08:00" and s.daytime_end == "20:00"
assert s.naver_headless is True
assert "new.land.naver.com" in s.naver_token_url
def test_override(monkeypatch):
monkeypatch.setenv("INTERNAL_API_KEY", "sekret")
monkeypatch.setenv("FETCH_INTERVAL_HOURS", "3")
monkeypatch.setenv("NAVER_HEADLESS", "0")
s = load_settings()
assert s.internal_api_key == "sekret"
assert s.fetch_interval_hours == 3.0
assert s.naver_headless is False
```
- [ ] **Step 3: 실패 확인**
Run: `python -m pytest services/naver-fetch/tests/test_config.py -q`
Expected: FAIL — `ModuleNotFoundError: No module named 'config'`
- [ ] **Step 4: config.py 구현**
```python
"""Settings — 환경변수 로드."""
from __future__ import annotations
import os
from dataclasses import dataclass
_DEFAULT_TOKEN_URL = (
"https://new.land.naver.com/houses?ms=37.489,126.921,16&a=APT:ABYG:JGC:PRE&e=RETAIL"
)
@dataclass
class Settings:
nas_base_url: str
internal_api_key: str
redis_url: str
fetch_interval_hours: float
daytime_start: str
daytime_end: str
naver_real_estate_type: str
naver_token_url: str
naver_headless: bool
def load_settings() -> Settings:
return Settings(
nas_base_url=os.getenv("NAS_BASE_URL", "http://192.168.45.54:18500"),
internal_api_key=os.getenv("INTERNAL_API_KEY", ""),
redis_url=os.getenv("REDIS_URL", "redis://192.168.45.54:6379"),
fetch_interval_hours=float(os.getenv("FETCH_INTERVAL_HOURS", "2.5")),
daytime_start=os.getenv("DAYTIME_START", "08:00"),
daytime_end=os.getenv("DAYTIME_END", "20:00"),
naver_real_estate_type=os.getenv("NAVER_REAL_ESTATE_TYPE", "APT:ABYG:JGC:PRE"),
naver_token_url=os.getenv("NAVER_TOKEN_URL", _DEFAULT_TOKEN_URL),
naver_headless=os.getenv("NAVER_HEADLESS", "1") == "1",
)
```
- [ ] **Step 5: 통과 확인**
Run: `python -m pytest services/naver-fetch/tests/test_config.py -q`
Expected: PASS (2 passed)
- [ ] **Step 6: 커밋**
```bash
git add services/naver-fetch/requirements.txt services/naver-fetch/conftest.py services/naver-fetch/pytest.ini services/naver-fetch/tests/__init__.py services/naver-fetch/config.py services/naver-fetch/tests/test_config.py services/naver-fetch/DESIGN.md services/naver-fetch/PLAN.md
git commit -m "feat(naver-fetch): 스캐폴딩 + config"
```
---
### Task 2: nas_client
**Files:**
- Create: `services/naver-fetch/nas_client.py`
- Test: `services/naver-fetch/tests/test_nas_client.py`
**Interfaces:**
- Produces `NASClient`:
- `__init__(base_url, api_key, timeout=15.0)`
- `async get_targets() -> dict` (GET `/api/internal/realestate/targets`)
- `async post_ingest(fetched_at: str, batches: list[dict]) -> dict` (POST `/api/internal/realestate/listings-ingest`)
- `async close()`
- [ ] **Step 1: 실패 테스트**`services/naver-fetch/tests/test_nas_client.py`
```python
"""NASClient — targets/ingest + X-Internal-Key (respx)."""
import json as _json
import httpx
import respx
from nas_client import NASClient
BASE = "http://nas.test"
@respx.mock
async def test_get_targets_sends_key():
route = respx.get(f"{BASE}/api/internal/realestate/targets").mock(
return_value=httpx.Response(200, json={
"dongs": [{"dong": "신대방동", "cortar_no": "1159010100"}],
"deal_types": ["전세", "매매"], "page_limit": 2}))
c = NASClient(BASE, "KEY")
tg = await c.get_targets()
assert tg["page_limit"] == 2
assert tg["dongs"][0]["cortar_no"] == "1159010100"
assert route.calls.last.request.headers["X-Internal-Key"] == "KEY"
await c.close()
@respx.mock
async def test_post_ingest_payload():
captured = {}
def _resp(request):
captured.update(_json.loads(request.content))
return httpx.Response(200, json={"received": 3, "new": 2, "matched": 1})
respx.post(f"{BASE}/api/internal/realestate/listings-ingest").mock(side_effect=_resp)
c = NASClient(BASE, "KEY")
batches = [{"dong": "신대방동", "articles": [{"articleNo": "1"}, {"articleNo": "2"}]}]
out = await c.post_ingest("2026-07-10T05:00:00Z", batches)
assert out == {"received": 3, "new": 2, "matched": 1}
assert captured["fetched_at"] == "2026-07-10T05:00:00Z"
assert captured["batches"] == batches
await c.close()
```
- [ ] **Step 2: 실패 확인**
Run: `python -m pytest services/naver-fetch/tests/test_nas_client.py -q`
Expected: FAIL — `ModuleNotFoundError: No module named 'nas_client'`
- [ ] **Step 3: nas_client.py 구현**
```python
"""NAS realestate 내부 계약 — X-Internal-Key + retry."""
from __future__ import annotations
import asyncio
import logging
import httpx
logger = logging.getLogger(__name__)
_MAX_ATTEMPTS = 3
_RETRY_STATUSES = {429, 500, 502, 503, 504}
class NASClient:
def __init__(self, base_url: str, api_key: str, timeout: float = 15.0):
self._base_url = base_url.rstrip("/")
self._api_key = api_key
self._client = httpx.AsyncClient(timeout=timeout)
async def close(self) -> None:
await self._client.aclose()
async def get_targets(self) -> dict:
return await self._request("GET", "/api/internal/realestate/targets")
async def post_ingest(self, fetched_at: str, batches: list[dict]) -> dict:
return await self._request(
"POST", "/api/internal/realestate/listings-ingest",
json={"fetched_at": fetched_at, "batches": batches})
async def _request(self, method: str, path: str, **kwargs) -> dict:
url = f"{self._base_url}{path}"
headers = {"X-Internal-Key": self._api_key}
for attempt in range(_MAX_ATTEMPTS):
try:
resp = await self._client.request(method, url, headers=headers, **kwargs)
if resp.status_code in _RETRY_STATUSES and attempt < _MAX_ATTEMPTS - 1:
await asyncio.sleep(2 ** attempt)
continue
resp.raise_for_status()
return resp.json()
except httpx.TimeoutException:
if attempt < _MAX_ATTEMPTS - 1:
await asyncio.sleep(2 ** attempt)
continue
raise
raise RuntimeError("retry exhausted")
```
- [ ] **Step 4: 통과 확인**
Run: `python -m pytest services/naver-fetch/tests/test_nas_client.py -q`
Expected: PASS (2 passed)
- [ ] **Step 5: 커밋**
```bash
git add services/naver-fetch/nas_client.py services/naver-fetch/tests/test_nas_client.py
git commit -m "feat(naver-fetch): NAS realestate 클라이언트 (targets/ingest)"
```
---
### Task 3: naver_client (Playwright 토큰 + httpx fetch)
**Files:**
- Create: `services/naver-fetch/naver_client.py`
- Test: `services/naver-fetch/tests/test_naver_client.py`
**Interfaces:**
- Produces `NaverClient`:
- `__init__(real_estate_type: str, token_url: str, headless: bool = True, timeout: float = 20.0)`
- `async acquire_token() -> None` — Playwright로 `self._token`(Bearer str), `self._cookies`(str) 설정. 실패 시 `RuntimeError`. (실브라우저 — 단위테스트 제외, spike/첫운영 검증)
- `async fetch_articles(cortar_no: str, page: int = 1) -> dict` — httpx GET, raw JSON dict. 토큰 없으면 `RuntimeError`.
- `async close()`
- [ ] **Step 1: 실패 테스트**`services/naver-fetch/tests/test_naver_client.py`
```python
"""NaverClient.fetch_articles — httpx 경로 (respx). acquire_token은 실브라우저라 제외."""
import httpx
import pytest
import respx
from naver_client import NaverClient
API = "https://new.land.naver.com/api/articles"
def _client():
return NaverClient(real_estate_type="APT:ABYG", token_url="https://x", headless=True)
async def test_fetch_requires_token():
c = _client()
with pytest.raises(RuntimeError):
await c.fetch_articles("1159010100", 1)
await c.close()
@respx.mock
async def test_fetch_articles_request_and_parse():
captured = {}
def _resp(request):
captured["auth"] = request.headers.get("authorization")
captured["cookie"] = request.headers.get("cookie")
captured["url"] = str(request.url)
return httpx.Response(200, json={
"isMoreData": True,
"articleList": [{"articleNo": "111", "articleName": "테스트아파트"}]})
respx.get(API).mock(side_effect=_resp)
c = _client()
c._token = "Bearer TESTJWT" # acquire_token 우회(단위테스트)
c._cookies = "NNB=abc; page_uid=xyz"
out = await c.fetch_articles("1159010100", page=2)
assert out["articleList"][0]["articleNo"] == "111"
assert captured["auth"] == "Bearer TESTJWT"
assert captured["cookie"] == "NNB=abc; page_uid=xyz"
assert "cortarNo=1159010100" in captured["url"]
assert "page=2" in captured["url"]
assert "realEstateType=APT" in captured["url"]
await c.close()
@respx.mock
async def test_fetch_non_200_raises():
respx.get(API).mock(return_value=httpx.Response(429, json={"code": "TOO_MANY_REQUESTS"}))
c = _client()
c._token = "Bearer X"
with pytest.raises(httpx.HTTPStatusError):
await c.fetch_articles("1159010100", 1)
await c.close()
```
- [ ] **Step 2: 실패 확인**
Run: `python -m pytest services/naver-fetch/tests/test_naver_client.py -q`
Expected: FAIL — `ModuleNotFoundError: No module named 'naver_client'`
- [ ] **Step 3: naver_client.py 구현**
```python
"""Naver land client — Playwright 토큰 획득 + httpx article fetch."""
from __future__ import annotations
import logging
import httpx
logger = logging.getLogger(__name__)
_UA = ("Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 "
"(KHTML, like Gecko) Chrome/130.0.0.0 Safari/537.36")
_API = "https://new.land.naver.com/api/articles"
class NaverClient:
def __init__(self, real_estate_type: str, token_url: str,
headless: bool = True, timeout: float = 20.0):
self._ret = real_estate_type
self._token_url = token_url
self._headless = headless
self._client = httpx.AsyncClient(timeout=timeout)
self._token: str | None = None
self._cookies: str = ""
async def close(self) -> None:
await self._client.aclose()
async def acquire_token(self) -> None:
"""new.land.naver.com 로드 → XHR Authorization 헤더 인터셉트."""
from playwright.async_api import async_playwright
captured: dict[str, str] = {}
async with async_playwright() as p:
browser = await p.chromium.launch(headless=self._headless)
try:
ctx = await browser.new_context(user_agent=_UA, locale="ko-KR")
page = await ctx.new_page()
def on_request(req):
auth = req.headers.get("authorization")
if auth and "bearer" in auth.lower() and "auth" not in captured:
captured["auth"] = auth
page.on("request", on_request)
try:
await page.goto(self._token_url, wait_until="networkidle", timeout=45000)
except Exception:
logger.warning("naver token goto 지연/부분 실패 — 캡처 계속")
await page.wait_for_timeout(4000)
cookies = await ctx.cookies()
self._cookies = "; ".join(f"{c['name']}={c['value']}" for c in cookies)
finally:
await browser.close()
token = captured.get("auth")
if not token:
raise RuntimeError("naver Bearer token 획득 실패")
self._token = token
logger.info("naver token 획득 (len=%d)", len(token))
async def fetch_articles(self, cortar_no: str, page: int = 1) -> dict:
if not self._token:
raise RuntimeError("token 없음 — acquire_token 먼저 호출")
params = {
"cortarNo": cortar_no, "order": "rank", "realEstateType": self._ret,
"tradeType": "", "rentPriceMin": 0, "rentPriceMax": 900000000,
"priceMin": 0, "priceMax": 900000000, "areaMin": 0, "areaMax": 900000000,
"showArticle": "false", "sameAddressGroup": "false",
"priceType": "RETAIL", "page": page,
}
headers = {
"authorization": self._token, "user-agent": _UA,
"referer": "https://new.land.naver.com/",
"accept": "application/json, text/plain, */*",
"accept-language": "ko-KR,ko;q=0.9", "cookie": self._cookies,
}
resp = await self._client.get(_API, params=params, headers=headers)
resp.raise_for_status()
return resp.json()
```
- [ ] **Step 4: 통과 확인**
Run: `python -m pytest services/naver-fetch/tests/test_naver_client.py -q`
Expected: PASS (3 passed)
- [ ] **Step 5: 커밋**
```bash
git add services/naver-fetch/naver_client.py services/naver-fetch/tests/test_naver_client.py
git commit -m "feat(naver-fetch): NaverClient (Playwright 토큰 + httpx fetch)"
```
---
### Task 4: fetch_worker (오케스트레이션 + heartbeat)
**Files:**
- Create: `services/naver-fetch/fetch_worker.py`
- Test: `services/naver-fetch/tests/test_fetch_worker.py`
**Interfaces:**
- Consumes: `NASClient`, `NaverClient`, `config.Settings`.
- Produces:
- `class FetchState``.state="idle"`, `.last_fetch_at=None`, `.listings_pushed=0`, `.errors=0`
- `in_daytime(now: datetime, start: str, end: str) -> bool`
- `async run_cycle(nas, naver, state, settings) -> None`
- `async fetch_loop(nas, naver, state, settings) -> None`
- `build_heartbeat(state) -> str` (JSON)
- `async heartbeat_loop(redis, state, interval=15, ttl=45) -> None`
- [ ] **Step 1: 실패 테스트**`services/naver-fetch/tests/test_fetch_worker.py`
```python
"""fetch_worker — 주간창/조립/격리/heartbeat."""
import datetime as dt
import json
from zoneinfo import ZoneInfo
from fetch_worker import FetchState, in_daytime, run_cycle, build_heartbeat
from config import load_settings
KST = ZoneInfo("Asia/Seoul")
def _kst(hh, mm):
return dt.datetime(2026, 7, 10, hh, mm, tzinfo=KST)
def test_in_daytime_boundaries():
assert in_daytime(_kst(8, 0), "08:00", "20:00") is True
assert in_daytime(_kst(19, 59), "08:00", "20:00") is True
assert in_daytime(_kst(20, 0), "08:00", "20:00") is False
assert in_daytime(_kst(7, 59), "08:00", "20:00") is False
class _FakeNAS:
def __init__(self, targets):
self._targets = targets
self.ingested = None
async def get_targets(self):
return self._targets
async def post_ingest(self, fetched_at, batches):
self.ingested = {"fetched_at": fetched_at, "batches": batches}
return {"received": sum(len(b["articles"]) for b in batches), "new": 1, "matched": 1}
class _FakeNaver:
def __init__(self, fail_dongs=None):
self._fail = fail_dongs or set()
self.token_acquired = False
async def acquire_token(self):
self.token_acquired = True
async def fetch_articles(self, cortar_no, page=1):
if cortar_no in self._fail:
raise RuntimeError("naver 429")
return {"isMoreData": False,
"articleList": [{"articleNo": f"{cortar_no}-{page}"}]}
async def test_run_cycle_assembles_batches_and_ingests():
nas = _FakeNAS({"dongs": [{"dong": "신대방동", "cortar_no": "1159010100"},
{"dong": "봉천동", "cortar_no": "1162010100"}],
"deal_types": ["매매"], "page_limit": 1})
naver = _FakeNaver()
state = FetchState()
await run_cycle(nas, naver, state, load_settings())
assert naver.token_acquired is True
assert nas.ingested is not None
dongs = [b["dong"] for b in nas.ingested["batches"]]
assert dongs == ["신대방동", "봉천동"]
assert state.listings_pushed == 2
assert state.last_fetch_at is not None
assert state.state == "idle"
assert nas.ingested["fetched_at"].endswith("Z")
async def test_run_cycle_dong_failure_isolated():
nas = _FakeNAS({"dongs": [{"dong": "A", "cortar_no": "1159010100"},
{"dong": "B", "cortar_no": "1162010100"}],
"page_limit": 1})
naver = _FakeNaver(fail_dongs={"1159010100"})
state = FetchState()
await run_cycle(nas, naver, state, load_settings())
assert nas.ingested is not None # 실패해도 ingest 진행
assert state.errors >= 1
batches = {b["dong"]: b["articles"] for b in nas.ingested["batches"]}
assert batches["A"] == [] # 실패 동 빈 배열
assert len(batches["B"]) == 1 # 성공 동
async def test_run_cycle_token_failure_skips_ingest():
class _BadNaver(_FakeNaver):
async def acquire_token(self):
raise RuntimeError("token 실패")
nas = _FakeNAS({"dongs": [], "page_limit": 1})
state = FetchState()
await run_cycle(nas, _BadNaver(), state, load_settings())
assert nas.ingested is None
assert state.errors == 1
assert state.state == "idle"
def test_build_heartbeat_schema():
state = FetchState()
state.state = "fetching"
state.listings_pushed = 7
d = json.loads(build_heartbeat(state))
assert d["name"] == "naver-fetch"
assert d["kind"] == "fetcher"
assert d["state"] == "fetching"
assert isinstance(d["ts"], int)
assert d["last_fetch_at"] is None
assert d["listings_pushed"] == 7
assert d["errors"] == 0
```
- [ ] **Step 2: 실패 확인**
Run: `python -m pytest services/naver-fetch/tests/test_fetch_worker.py -q`
Expected: FAIL — `ModuleNotFoundError: No module named 'fetch_worker'`
- [ ] **Step 3: fetch_worker.py 구현**
```python
"""오케스트레이션 — 주간 스케줄 fetch + fetcher heartbeat."""
from __future__ import annotations
import asyncio
import datetime as dt
import json
import logging
from zoneinfo import ZoneInfo
logger = logging.getLogger(__name__)
KST = ZoneInfo("Asia/Seoul")
HEARTBEAT_KEY = "worker:naver-fetch:heartbeat"
class FetchState:
"""fetch_loop가 갱신, heartbeat_loop가 읽는 공유 상태."""
def __init__(self):
self.state = "idle" # idle | fetching
self.last_fetch_at: int | None = None
self.listings_pushed = 0
self.errors = 0
def _parse_hhmm(s: str) -> dt.time:
hh, mm = s.split(":")
return dt.time(int(hh), int(mm))
def in_daytime(now: dt.datetime, start: str, end: str) -> bool:
t = now.timetz().replace(tzinfo=None)
return _parse_hhmm(start) <= t < _parse_hhmm(end)
def _epoch_now() -> int:
return int(dt.datetime.now(dt.timezone.utc).timestamp())
def _iso_utc_now() -> str:
return dt.datetime.now(dt.timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ")
async def run_cycle(nas, naver, state, settings) -> None:
state.state = "fetching"
try:
await naver.acquire_token()
except Exception:
logger.exception("naver 토큰 획득 실패")
state.errors += 1
state.state = "idle"
return
try:
tg = await nas.get_targets()
except Exception:
logger.exception("targets 조회 실패")
state.errors += 1
state.state = "idle"
return
dongs = tg.get("dongs", [])
page_limit = int(tg.get("page_limit", 1))
batches: list[dict] = []
for d in dongs:
cortar = d.get("cortar_no")
dong = d.get("dong")
articles: list[dict] = []
try:
for page in range(1, page_limit + 1):
resp = await naver.fetch_articles(cortar, page)
articles += resp.get("articleList", [])
if not resp.get("isMoreData"):
break
except Exception:
logger.exception("naver fetch 실패 dong=%s", dong)
state.errors += 1
batches.append({"dong": dong, "articles": articles})
fetched_at = _iso_utc_now()
try:
r = await nas.post_ingest(fetched_at, batches)
logger.info("ingest received=%s new=%s matched=%s",
r.get("received"), r.get("new"), r.get("matched"))
except Exception:
logger.exception("listings-ingest 실패")
state.errors += 1
state.state = "idle"
return
state.listings_pushed += sum(len(b["articles"]) for b in batches)
state.last_fetch_at = _epoch_now()
state.state = "idle"
async def fetch_loop(nas, naver, state, settings) -> None:
interval = dt.timedelta(hours=settings.fetch_interval_hours)
last_run: dt.datetime | None = None
logger.info("naver-fetch loop 시작 interval=%.1fh 주간창 %s-%s",
settings.fetch_interval_hours, settings.daytime_start, settings.daytime_end)
while True:
try:
now = dt.datetime.now(KST)
if in_daytime(now, settings.daytime_start, settings.daytime_end) and (
last_run is None or (now - last_run) >= interval):
await run_cycle(nas, naver, state, settings)
last_run = now
except asyncio.CancelledError:
logger.info("fetch_loop cancelled")
raise
except Exception:
logger.exception("fetch_loop iteration 실패")
await asyncio.sleep(300)
def build_heartbeat(state) -> str:
return json.dumps({
"name": "naver-fetch", "kind": "fetcher", "state": state.state,
"ts": _epoch_now(), "last_fetch_at": state.last_fetch_at,
"listings_pushed": state.listings_pushed, "errors": state.errors,
})
async def heartbeat_loop(redis, state, interval: int = 15, ttl: int = 45) -> None:
logger.info("naver-fetch heartbeat 시작 ttl=%ds", ttl)
while True:
try:
await redis.set(HEARTBEAT_KEY, build_heartbeat(state), ex=ttl)
except asyncio.CancelledError:
raise
except Exception:
logger.exception("heartbeat 발신 실패")
await asyncio.sleep(interval)
```
- [ ] **Step 4: 통과 확인**
Run: `python -m pytest services/naver-fetch/tests/test_fetch_worker.py -q`
Expected: PASS (5 passed)
- [ ] **Step 5: 커밋**
```bash
git add services/naver-fetch/fetch_worker.py services/naver-fetch/tests/test_fetch_worker.py
git commit -m "feat(naver-fetch): fetch_worker (run_cycle/loop/heartbeat)"
```
---
### Task 5: main.py (FastAPI lifespan + /health)
**Files:**
- Create: `services/naver-fetch/main.py`
- Test: `services/naver-fetch/tests/test_health.py`
**Interfaces:**
- Consumes: `config.load_settings`, `NASClient`, `NaverClient`, `fetch_worker.*`, `redis.asyncio`.
- Produces: FastAPI `app` + `health()``{"ok": True, "service": "naver-fetch"}`.
- [ ] **Step 1: 실패 테스트**`services/naver-fetch/tests/test_health.py`
```python
"""/health — 핸들러 직접 검증."""
from main import health
def test_health():
assert health() == {"ok": True, "service": "naver-fetch"}
```
- [ ] **Step 2: 실패 확인**
Run: `python -m pytest services/naver-fetch/tests/test_health.py -q`
Expected: FAIL — `ModuleNotFoundError: No module named 'main'`
- [ ] **Step 3: main.py 구현**
```python
"""naver-fetch FastAPI entry — lifespan(fetch_loop + heartbeat_loop) + /health."""
from __future__ import annotations
import asyncio
import logging
from contextlib import asynccontextmanager
import redis.asyncio as aioredis
from fastapi import FastAPI
import fetch_worker
from config import load_settings
from nas_client import NASClient
from naver_client import NaverClient
logging.basicConfig(level=logging.INFO,
format="%(asctime)s %(name)s %(levelname)s %(message)s")
logger = logging.getLogger(__name__)
@asynccontextmanager
async def lifespan(app: FastAPI):
settings = load_settings()
nas = NASClient(settings.nas_base_url, settings.internal_api_key)
naver = NaverClient(settings.naver_real_estate_type, settings.naver_token_url,
settings.naver_headless)
state = fetch_worker.FetchState()
redis = aioredis.from_url(settings.redis_url, decode_responses=False)
ft = asyncio.create_task(fetch_worker.fetch_loop(nas, naver, state, settings))
hb = asyncio.create_task(fetch_worker.heartbeat_loop(redis, state))
logger.info("naver-fetch lifespan 시작")
try:
yield
finally:
for t in (ft, hb):
t.cancel()
try:
await t
except asyncio.CancelledError:
pass
await naver.close()
await nas.close()
await redis.aclose()
logger.info("naver-fetch lifespan 종료")
app = FastAPI(lifespan=lifespan)
@app.get("/health")
def health():
return {"ok": True, "service": "naver-fetch"}
```
- [ ] **Step 4: 통과 확인 + 전체 스위트**
Run: `python -m pytest services/naver-fetch/tests -q`
Expected: PASS (config 2 + nas 2 + naver 3 + fetch_worker 5 + health 1 = 13)
- [ ] **Step 5: 커밋**
```bash
git add services/naver-fetch/main.py services/naver-fetch/tests/test_health.py
git commit -m "feat(naver-fetch): FastAPI lifespan + heartbeat 배선 + /health"
```
---
### Task 6: 배포 (Dockerfile + compose + .env.example)
**Files:**
- Create: `services/naver-fetch/Dockerfile`, `services/naver-fetch/.env.example`
- Modify: `services/docker-compose.yml`
**Interfaces:** 없음(배포). 검증 `docker compose config`.
- [ ] **Step 1: Dockerfile**`services/naver-fetch/Dockerfile` (insta-render Chromium 관례 복제)
```dockerfile
FROM python:3.12-slim-bookworm
ENV PYTHONUNBUFFERED=1
WORKDIR /app
# Chromium 런타임 deps (Debian 12 / bookworm) — insta-render 관례
RUN apt-get update && apt-get install -y --no-install-recommends \
ca-certificates tzdata fonts-noto-cjk \
libnss3 libnspr4 libdbus-1-3 libatk1.0-0 libatk-bridge2.0-0 \
libcups2 libdrm2 libxkbcommon0 libxcomposite1 libxdamage1 \
libxfixes3 libxrandr2 libgbm1 libxshmfence1 libpango-1.0-0 \
libcairo2 libasound2 libatspi2.0-0 \
&& rm -rf /var/lib/apt/lists/*
COPY naver-fetch/requirements.txt /app/
RUN pip install --no-cache-dir --timeout 600 --retries 5 -r requirements.txt
RUN playwright install chromium
# 공통 heartbeat 등 모듈 (services/_shared) — 빌드 컨텍스트 . 필요
COPY _shared /app/_shared
COPY naver-fetch/. /app/
ENV PYTHONPATH=/app
EXPOSE 8000
CMD ["python", "-m", "uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000", "--workers", "1"]
```
- [ ] **Step 2: .env.example**`services/naver-fetch/.env.example`
```
# naver-fetch — realestate 매물 fetch 워커
REDIS_URL=redis://192.168.45.54:6379
NAS_BASE_URL=http://192.168.45.54:18500
# render 워커 공유키 (X-Internal-Key)
INTERNAL_API_KEY=
# 스케줄 (KST)
FETCH_INTERVAL_HOURS=2.5
DAYTIME_START=08:00
DAYTIME_END=20:00
# 네이버
NAVER_REAL_ESTATE_TYPE=APT:ABYG:JGC:PRE
NAVER_HEADLESS=1
# NAVER_TOKEN_URL= # 기본값(new.land.naver.com houses) 사용
```
- [ ] **Step 3: docker-compose.yml에 서비스 추가**
`services/docker-compose.yml``trade-monitor` 블록 뒤(파일 끝, 들여쓰기 2칸)에 추가:
```yaml
naver-fetch:
build:
context: .
dockerfile: naver-fetch/Dockerfile
container_name: naver-fetch
restart: unless-stopped
ports:
- "18716:8000"
environment:
- TZ=Asia/Seoul
- REDIS_URL=${REDIS_URL:-redis://192.168.45.54:6379}
- NAS_BASE_URL=${NAS_BASE_URL:-http://192.168.45.54:18500}
- INTERNAL_API_KEY=${INTERNAL_API_KEY:-}
- FETCH_INTERVAL_HOURS=${FETCH_INTERVAL_HOURS:-2.5}
- DAYTIME_START=${DAYTIME_START:-08:00}
- DAYTIME_END=${DAYTIME_END:-20:00}
- NAVER_REAL_ESTATE_TYPE=${NAVER_REAL_ESTATE_TYPE:-APT:ABYG:JGC:PRE}
- NAVER_HEADLESS=${NAVER_HEADLESS:-1}
healthcheck:
test: ["CMD", "python", "-c", "import urllib.request; urllib.request.urlopen('http://localhost:8000/health')"]
interval: 60s
timeout: 5s
retries: 3
```
- [ ] **Step 4: compose 검증**
Run: `cd services && docker compose config` (실패 시 YAML 들여쓰기 육안 확인)
Expected: `naver-fetch` 서비스 파싱, 포트 18716.
- [ ] **Step 5: 커밋**
```bash
git add services/naver-fetch/Dockerfile services/naver-fetch/.env.example services/docker-compose.yml
git commit -m "feat(naver-fetch): Dockerfile(Playwright) + compose 서비스(18716) + .env.example"
```
---
## Self-Review
**1. Spec coverage:**
- §4 `GET /targets` → Task 2 `get_targets` + Task 4 사용. ✅
- §4 `POST /listings-ingest`(fetched_at ISO, batches) → Task 2 `post_ingest` + Task 4 조립. ✅
- §4.4 heartbeat(EX45, kind=fetcher, epoch ts, 필드) → Task 4 `build_heartbeat`/`heartbeat_loop`. ✅
- 워커 루프(targets→네이버 fetch→ingest) → Task 4 `run_cycle`. ✅
- 네이버 Bearer 토큰(Playwright) + fetch → Task 3. ✅
- 스케줄(주간 2~3h) → Task 4 `in_daytime` + `fetch_loop`. ✅
- 얇은 프록시(raw 전달) → Task 4(파싱 없음, articleList 그대로). ✅
- 배포(_shared COPY, Playwright, 18716) → Task 6. ✅
**2. Placeholder scan:** 실제 코드/명령/기대출력 기재. TODO 없음. ✅
**3. Type consistency:** `FetchState`(state/last_fetch_at/listings_pushed/errors)가 Task 4 생성 ↔ `build_heartbeat` 소비 일치. `NaverClient`(acquire_token/fetch_articles) Task 3 정의 ↔ Task 4 `run_cycle` 호출 일치. `NASClient`(get_targets/post_ingest) Task 2 ↔ Task 4 일치. fetched_at ISO 문자열/heartbeat ts epoch 구분 유지. ✅
**미해결 플래그(DESIGN §11):** INTERNAL_API_KEY vs REALESTATE_INTERNAL_KEY(배포 전 BE 확인), 헤드리스 탐지, deal_types 미사용(NAS 필터), 토큰 만료 재획득.

View File

@@ -0,0 +1,36 @@
"""Settings — 환경변수 로드."""
from __future__ import annotations
import os
from dataclasses import dataclass
_DEFAULT_TOKEN_URL = (
"https://new.land.naver.com/houses?ms=37.489,126.921,16&a=APT:ABYG:JGC:PRE&e=RETAIL"
)
@dataclass
class Settings:
nas_base_url: str
internal_api_key: str
redis_url: str
fetch_interval_hours: float
daytime_start: str
daytime_end: str
naver_real_estate_type: str
naver_token_url: str
naver_headless: bool
def load_settings() -> Settings:
return Settings(
nas_base_url=os.getenv("NAS_BASE_URL", "http://192.168.45.54:18800"),
internal_api_key=os.getenv("INTERNAL_API_KEY", ""),
redis_url=os.getenv("REDIS_URL", "redis://192.168.45.54:6379"),
fetch_interval_hours=float(os.getenv("FETCH_INTERVAL_HOURS", "2.5")),
daytime_start=os.getenv("DAYTIME_START", "08:00"),
daytime_end=os.getenv("DAYTIME_END", "20:00"),
naver_real_estate_type=os.getenv("NAVER_REAL_ESTATE_TYPE", "APT:ABYG:JGC:PRE"),
naver_token_url=os.getenv("NAVER_TOKEN_URL", _DEFAULT_TOKEN_URL),
naver_headless=os.getenv("NAVER_HEADLESS", "1") == "1",
)

View File

@@ -0,0 +1,5 @@
"""services/ 루트를 sys.path에 추가 — from _shared... import 가능하게."""
import sys
from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))

View File

@@ -0,0 +1,129 @@
"""오케스트레이션 — 주간 스케줄 fetch + fetcher heartbeat."""
from __future__ import annotations
import asyncio
import datetime as dt
import json
import logging
from zoneinfo import ZoneInfo
logger = logging.getLogger(__name__)
KST = ZoneInfo("Asia/Seoul")
HEARTBEAT_KEY = "worker:naver-fetch:heartbeat"
class FetchState:
"""fetch_loop가 갱신, heartbeat_loop가 읽는 공유 상태."""
def __init__(self):
self.state = "idle" # idle | fetching
self.last_fetch_at: int | None = None
self.listings_pushed = 0
self.errors = 0
def _parse_hhmm(s: str) -> dt.time:
hh, mm = s.split(":")
return dt.time(int(hh), int(mm))
def in_daytime(now: dt.datetime, start: str, end: str) -> bool:
t = now.timetz().replace(tzinfo=None)
return _parse_hhmm(start) <= t < _parse_hhmm(end)
def _epoch_now() -> int:
return int(dt.datetime.now(dt.timezone.utc).timestamp())
def _iso_utc_now() -> str:
return dt.datetime.now(dt.timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ")
async def run_cycle(nas, naver, state, settings) -> None:
state.state = "fetching"
try:
await naver.acquire_token()
except Exception:
logger.exception("naver 토큰 획득 실패")
state.errors += 1
state.state = "idle"
return
try:
tg = await nas.get_targets()
except Exception:
logger.exception("targets 조회 실패")
state.errors += 1
state.state = "idle"
return
dongs = tg.get("dongs", [])
page_limit = int(tg.get("page_limit", 1))
fetched_at = _iso_utc_now()
# 동별로 fetch → 즉시 ingest. 전체 batch를 한 POST로 보내면 NAS 동기 처리(매물 수 비례)가
# >180s로 타임아웃하므로 동 단위로 쪼갬(각 ~수십건 → 빠름, 부분 실패 격리). batches 리스트 계약 호환.
for d in dongs:
cortar = d.get("cortar_no")
dong = d.get("dong")
try:
articles: list[dict] = []
for page in range(1, page_limit + 1):
resp = await naver.fetch_articles(cortar, page)
articles += resp.get("articleList", [])
if not resp.get("isMoreData"):
break
except Exception:
logger.exception("naver fetch 실패 dong=%s", dong)
state.errors += 1
continue
try:
r = await nas.post_ingest(fetched_at, [{"dong": dong, "articles": articles}])
# BE ingest async화(main 8e28ce9): 응답 {received,new,queued} (matched는 백그라운드라 제거됨)
logger.info("ingest dong=%s received=%s new=%s queued=%s",
dong, r.get("received"), r.get("new"), r.get("queued"))
state.listings_pushed += len(articles)
except Exception:
logger.exception("listings-ingest 실패 dong=%s", dong)
state.errors += 1
state.last_fetch_at = _epoch_now()
state.state = "idle"
async def fetch_loop(nas, naver, state, settings) -> None:
interval = dt.timedelta(hours=settings.fetch_interval_hours)
last_run: dt.datetime | None = None
logger.info("naver-fetch loop 시작 interval=%.1fh 주간창 %s-%s",
settings.fetch_interval_hours, settings.daytime_start, settings.daytime_end)
while True:
try:
now = dt.datetime.now(KST)
if in_daytime(now, settings.daytime_start, settings.daytime_end) and (
last_run is None or (now - last_run) >= interval):
await run_cycle(nas, naver, state, settings)
last_run = now
except asyncio.CancelledError:
logger.info("fetch_loop cancelled")
raise
except Exception:
logger.exception("fetch_loop iteration 실패")
await asyncio.sleep(300)
def build_heartbeat(state) -> str:
return json.dumps({
"name": "naver-fetch", "kind": "fetcher", "state": state.state,
"ts": _epoch_now(), "last_fetch_at": state.last_fetch_at,
"listings_pushed": state.listings_pushed, "errors": state.errors,
})
async def heartbeat_loop(redis, state, interval: int = 15, ttl: int = 45) -> None:
logger.info("naver-fetch heartbeat 시작 ttl=%ds", ttl)
while True:
try:
await redis.set(HEARTBEAT_KEY, build_heartbeat(state), ex=ttl)
except asyncio.CancelledError:
raise
except Exception:
logger.exception("heartbeat 발신 실패")
await asyncio.sleep(interval)

View File

@@ -0,0 +1,53 @@
"""naver-fetch FastAPI entry — lifespan(fetch_loop + heartbeat_loop) + /health."""
from __future__ import annotations
import asyncio
import logging
from contextlib import asynccontextmanager
import redis.asyncio as aioredis
from fastapi import FastAPI
import fetch_worker
from config import load_settings
from nas_client import NASClient
from naver_client import NaverClient
logging.basicConfig(level=logging.INFO,
format="%(asctime)s %(name)s %(levelname)s %(message)s")
logger = logging.getLogger(__name__)
@asynccontextmanager
async def lifespan(app: FastAPI):
settings = load_settings()
nas = NASClient(settings.nas_base_url, settings.internal_api_key)
naver = NaverClient(settings.naver_real_estate_type, settings.naver_token_url,
settings.naver_headless)
state = fetch_worker.FetchState()
redis = aioredis.from_url(settings.redis_url, decode_responses=False)
ft = asyncio.create_task(fetch_worker.fetch_loop(nas, naver, state, settings))
hb = asyncio.create_task(fetch_worker.heartbeat_loop(redis, state))
logger.info("naver-fetch lifespan 시작")
try:
yield
finally:
for t in (ft, hb):
t.cancel()
try:
await t
except asyncio.CancelledError:
pass
await naver.close()
await nas.close()
await redis.aclose()
logger.info("naver-fetch lifespan 종료")
app = FastAPI(lifespan=lifespan)
@app.get("/health")
def health():
return {"ok": True, "service": "naver-fetch"}

View File

@@ -0,0 +1,55 @@
"""NAS realestate 내부 계약 — X-Internal-Key + retry."""
from __future__ import annotations
import asyncio
import logging
import httpx
logger = logging.getLogger(__name__)
_RETRY_STATUSES = {429, 500, 502, 503, 504}
_TARGETS_TIMEOUT = 15.0
_INGEST_TIMEOUT = 180.0 # 첫 실행 bulk upsert/매칭/알림이 느림 (초 단위 수십)
class NASClient:
def __init__(self, base_url: str, api_key: str):
self._base_url = base_url.rstrip("/")
self._api_key = api_key
self._client = httpx.AsyncClient()
async def close(self) -> None:
await self._client.aclose()
async def get_targets(self) -> dict:
return await self._request("GET", "/api/internal/realestate/targets",
timeout=_TARGETS_TIMEOUT, max_attempts=3)
async def post_ingest(self, fetched_at: str, batches: list[dict]) -> dict:
# ingest는 재시도 안 함(max_attempts=1) — 타임아웃 후 재전송 시
# NAS가 같은 batch를 재처리/재알림할 위험. 실패 시 다음 사이클에 재시도.
return await self._request(
"POST", "/api/internal/realestate/listings-ingest",
json={"fetched_at": fetched_at, "batches": batches},
timeout=_INGEST_TIMEOUT, max_attempts=1)
async def _request(self, method: str, path: str, *, timeout: float,
max_attempts: int, **kwargs) -> dict:
url = f"{self._base_url}{path}"
headers = {"X-Internal-Key": self._api_key}
for attempt in range(max_attempts):
try:
resp = await self._client.request(method, url, headers=headers,
timeout=timeout, **kwargs)
if resp.status_code in _RETRY_STATUSES and attempt < max_attempts - 1:
await asyncio.sleep(2 ** attempt)
continue
resp.raise_for_status()
return resp.json()
except httpx.TimeoutException:
if attempt < max_attempts - 1:
await asyncio.sleep(2 ** attempt)
continue
raise
raise RuntimeError("retry exhausted")

View File

@@ -0,0 +1,79 @@
"""Naver land client — Playwright 토큰 획득 + httpx article fetch."""
from __future__ import annotations
import logging
import httpx
logger = logging.getLogger(__name__)
_UA = ("Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 "
"(KHTML, like Gecko) Chrome/130.0.0.0 Safari/537.36")
_API = "https://new.land.naver.com/api/articles"
class NaverClient:
def __init__(self, real_estate_type: str, token_url: str,
headless: bool = True, timeout: float = 20.0):
self._ret = real_estate_type
self._token_url = token_url
self._headless = headless
self._client = httpx.AsyncClient(timeout=timeout)
self._token: str | None = None
self._cookies: str = ""
async def close(self) -> None:
await self._client.aclose()
async def acquire_token(self) -> None:
"""new.land.naver.com 로드 → XHR Authorization 헤더 인터셉트."""
from playwright.async_api import async_playwright
captured: dict[str, str] = {}
async with async_playwright() as p:
browser = await p.chromium.launch(headless=self._headless)
try:
ctx = await browser.new_context(user_agent=_UA, locale="ko-KR")
page = await ctx.new_page()
def on_request(req):
auth = req.headers.get("authorization")
if auth and "bearer" in auth.lower() and "auth" not in captured:
captured["auth"] = auth
page.on("request", on_request)
try:
await page.goto(self._token_url, wait_until="networkidle", timeout=45000)
except Exception:
logger.warning("naver token goto 지연/부분 실패 — 캡처 계속")
await page.wait_for_timeout(4000)
cookies = await ctx.cookies()
self._cookies = "; ".join(f"{c['name']}={c['value']}" for c in cookies)
finally:
await browser.close()
token = captured.get("auth")
if not token:
raise RuntimeError("naver Bearer token 획득 실패")
self._token = token
logger.info("naver token 획득 (len=%d)", len(token))
async def fetch_articles(self, cortar_no: str, page: int = 1) -> dict:
if not self._token:
raise RuntimeError("token 없음 — acquire_token 먼저 호출")
params = {
"cortarNo": cortar_no, "order": "rank", "realEstateType": self._ret,
"tradeType": "", "rentPriceMin": 0, "rentPriceMax": 900000000,
"priceMin": 0, "priceMax": 900000000, "areaMin": 0, "areaMax": 900000000,
"showArticle": "false", "sameAddressGroup": "false",
"priceType": "RETAIL", "page": page,
}
headers = {
"authorization": self._token, "user-agent": _UA,
"referer": "https://new.land.naver.com/",
"accept": "application/json, text/plain, */*",
"accept-language": "ko-KR,ko;q=0.9", "cookie": self._cookies,
}
resp = await self._client.get(_API, params=params, headers=headers)
resp.raise_for_status()
return resp.json()

View File

@@ -0,0 +1,2 @@
[pytest]
asyncio_mode = auto

View File

@@ -0,0 +1,8 @@
fastapi==0.115.6
uvicorn[standard]==0.34.0
playwright==1.48.0
redis>=5.0
httpx>=0.27
pytest>=8.0
pytest-asyncio>=0.24
respx>=0.21

View File

View File

@@ -0,0 +1,25 @@
"""Settings env 로드."""
from config import load_settings
def test_defaults(monkeypatch):
for k in ("NAS_BASE_URL", "INTERNAL_API_KEY", "REDIS_URL", "FETCH_INTERVAL_HOURS",
"DAYTIME_START", "DAYTIME_END", "NAVER_REAL_ESTATE_TYPE",
"NAVER_TOKEN_URL", "NAVER_HEADLESS"):
monkeypatch.delenv(k, raising=False)
s = load_settings()
assert s.nas_base_url == "http://192.168.45.54:18800"
assert s.fetch_interval_hours == 2.5
assert s.daytime_start == "08:00" and s.daytime_end == "20:00"
assert s.naver_headless is True
assert "new.land.naver.com" in s.naver_token_url
def test_override(monkeypatch):
monkeypatch.setenv("INTERNAL_API_KEY", "sekret")
monkeypatch.setenv("FETCH_INTERVAL_HOURS", "3")
monkeypatch.setenv("NAVER_HEADLESS", "0")
s = load_settings()
assert s.internal_api_key == "sekret"
assert s.fetch_interval_hours == 3.0
assert s.naver_headless is False

View File

@@ -0,0 +1,107 @@
"""fetch_worker — 주간창/조립/격리/heartbeat."""
import datetime as dt
import json
from zoneinfo import ZoneInfo
from fetch_worker import FetchState, in_daytime, run_cycle, build_heartbeat
from config import load_settings
KST = ZoneInfo("Asia/Seoul")
def _kst(hh, mm):
return dt.datetime(2026, 7, 10, hh, mm, tzinfo=KST)
def test_in_daytime_boundaries():
assert in_daytime(_kst(8, 0), "08:00", "20:00") is True
assert in_daytime(_kst(19, 59), "08:00", "20:00") is True
assert in_daytime(_kst(20, 0), "08:00", "20:00") is False
assert in_daytime(_kst(7, 59), "08:00", "20:00") is False
class _FakeNAS:
def __init__(self, targets):
self._targets = targets
self.ingests = [] # 동별 post_ingest 호출 기록
async def get_targets(self):
return self._targets
async def post_ingest(self, fetched_at, batches):
self.ingests.append({"fetched_at": fetched_at, "batches": batches})
n = sum(len(b["articles"]) for b in batches)
return {"received": n, "new": n, "queued": True}
class _FakeNaver:
def __init__(self, fail_dongs=None):
self._fail = fail_dongs or set()
self.token_acquired = False
async def acquire_token(self):
self.token_acquired = True
async def fetch_articles(self, cortar_no, page=1):
if cortar_no in self._fail:
raise RuntimeError("naver 429")
return {"isMoreData": False,
"articleList": [{"articleNo": f"{cortar_no}-{page}"}]}
async def test_run_cycle_ingests_per_dong():
nas = _FakeNAS({"dongs": [{"dong": "신대방동", "cortar_no": "1159010100"},
{"dong": "봉천동", "cortar_no": "1162010100"}],
"deal_types": ["매매"], "page_limit": 1})
naver = _FakeNaver()
state = FetchState()
await run_cycle(nas, naver, state, load_settings())
assert naver.token_acquired is True
assert len(nas.ingests) == 2 # 동별 1 POST씩
dongs = [ing["batches"][0]["dong"] for ing in nas.ingests]
assert dongs == ["신대방동", "봉천동"]
assert all(len(ing["batches"]) == 1 for ing in nas.ingests)
assert state.listings_pushed == 2
assert state.last_fetch_at is not None
assert state.state == "idle"
assert nas.ingests[0]["fetched_at"].endswith("Z")
async def test_run_cycle_dong_failure_isolated():
nas = _FakeNAS({"dongs": [{"dong": "A", "cortar_no": "1159010100"},
{"dong": "B", "cortar_no": "1162010100"}],
"page_limit": 1})
naver = _FakeNaver(fail_dongs={"1159010100"})
state = FetchState()
await run_cycle(nas, naver, state, load_settings())
# A는 fetch 실패 → ingest skip, B만 ingest (부분 실패 격리)
assert len(nas.ingests) == 1
assert nas.ingests[0]["batches"][0]["dong"] == "B"
assert state.errors >= 1
async def test_run_cycle_token_failure_skips_ingest():
class _BadNaver(_FakeNaver):
async def acquire_token(self):
raise RuntimeError("token 실패")
nas = _FakeNAS({"dongs": [], "page_limit": 1})
state = FetchState()
await run_cycle(nas, _BadNaver(), state, load_settings())
assert nas.ingests == []
assert state.errors == 1
assert state.state == "idle"
def test_build_heartbeat_schema():
state = FetchState()
state.state = "fetching"
state.listings_pushed = 7
d = json.loads(build_heartbeat(state))
assert d["name"] == "naver-fetch"
assert d["kind"] == "fetcher"
assert d["state"] == "fetching"
assert isinstance(d["ts"], int)
assert d["last_fetch_at"] is None
assert d["listings_pushed"] == 7
assert d["errors"] == 0

View File

@@ -0,0 +1,6 @@
"""/health — 핸들러 직접 검증."""
from main import health
def test_health():
assert health() == {"ok": True, "service": "naver-fetch"}

View File

@@ -0,0 +1,41 @@
"""NASClient — targets/ingest + X-Internal-Key (respx)."""
import json as _json
import httpx
import respx
from nas_client import NASClient
BASE = "http://nas.test"
@respx.mock
async def test_get_targets_sends_key():
route = respx.get(f"{BASE}/api/internal/realestate/targets").mock(
return_value=httpx.Response(200, json={
"dongs": [{"dong": "신대방동", "cortar_no": "1159010100"}],
"deal_types": ["전세", "매매"], "page_limit": 2}))
c = NASClient(BASE, "KEY")
tg = await c.get_targets()
assert tg["page_limit"] == 2
assert tg["dongs"][0]["cortar_no"] == "1159010100"
assert route.calls.last.request.headers["X-Internal-Key"] == "KEY"
await c.close()
@respx.mock
async def test_post_ingest_payload():
captured = {}
def _resp(request):
captured.update(_json.loads(request.content))
return httpx.Response(200, json={"received": 3, "new": 2, "matched": 1})
respx.post(f"{BASE}/api/internal/realestate/listings-ingest").mock(side_effect=_resp)
c = NASClient(BASE, "KEY")
batches = [{"dong": "신대방동", "articles": [{"articleNo": "1"}, {"articleNo": "2"}]}]
out = await c.post_ingest("2026-07-10T05:00:00Z", batches)
assert out == {"received": 3, "new": 2, "matched": 1}
assert captured["fetched_at"] == "2026-07-10T05:00:00Z"
assert captured["batches"] == batches
await c.close()

View File

@@ -0,0 +1,55 @@
"""NaverClient.fetch_articles — httpx 경로 (respx). acquire_token은 실브라우저라 제외."""
import httpx
import pytest
import respx
from naver_client import NaverClient
API = "https://new.land.naver.com/api/articles"
def _client():
return NaverClient(real_estate_type="APT:ABYG", token_url="https://x", headless=True)
async def test_fetch_requires_token():
c = _client()
with pytest.raises(RuntimeError):
await c.fetch_articles("1159010100", 1)
await c.close()
@respx.mock
async def test_fetch_articles_request_and_parse():
captured = {}
def _resp(request):
captured["auth"] = request.headers.get("authorization")
captured["cookie"] = request.headers.get("cookie")
captured["url"] = str(request.url)
return httpx.Response(200, json={
"isMoreData": True,
"articleList": [{"articleNo": "111", "articleName": "테스트아파트"}]})
respx.get(API).mock(side_effect=_resp)
c = _client()
c._token = "Bearer TESTJWT" # acquire_token 우회(단위테스트)
c._cookies = "NNB=abc; page_uid=xyz"
out = await c.fetch_articles("1159010100", page=2)
assert out["articleList"][0]["articleNo"] == "111"
assert captured["auth"] == "Bearer TESTJWT"
assert captured["cookie"] == "NNB=abc; page_uid=xyz"
assert "cortarNo=1159010100" in captured["url"]
assert "page=2" in captured["url"]
assert "realEstateType=APT" in captured["url"]
await c.close()
@respx.mock
async def test_fetch_non_200_raises():
respx.get(API).mock(return_value=httpx.Response(429, json={"code": "TOO_MANY_REQUESTS"}))
c = _client()
c._token = "Bearer X"
with pytest.raises(httpx.HTTPStatusError):
await c.fetch_articles("1159010100", 1)
await c.close()

View File

@@ -7,10 +7,13 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
ca-certificates tzdata \
&& rm -rf /var/lib/apt/lists/*
COPY requirements.txt .
COPY task-watcher/requirements.txt /app/
RUN pip install --no-cache-dir --timeout 600 --retries 5 -r requirements.txt
COPY . .
# 공통 heartbeat 모듈 (services/_shared) — watcher.py가 from _shared.heartbeat import
COPY _shared /app/_shared
COPY task-watcher/. /app/
ENV PYTHONPATH=/app
EXPOSE 8000
CMD ["python", "-m", "uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000", "--workers", "1"]

View File

@@ -0,0 +1,5 @@
"""Make services/ root importable so `from _shared.heartbeat import ...` works during tests."""
import sys
from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))

View File

@@ -0,0 +1,16 @@
"""task-watcher heartbeat payload — state=mode + mode 필드 검증."""
import json
from _shared.heartbeat import build_payload, WorkerStats
def test_watcher_heartbeat_payload_carries_mode():
payload = json.loads(
build_payload(
"task-watcher", "watcher", "trading",
WorkerStats(), extra={"mode": "trading"},
)
)
assert payload["kind"] == "watcher"
assert payload["state"] == "trading"
assert payload["mode"] == "trading"

View File

@@ -15,6 +15,7 @@ from zoneinfo import ZoneInfo
import redis.asyncio as aioredis
from mode import current_mode, fetch_holidays, KST
from _shared.heartbeat import build_payload, WorkerStats
logger = logging.getLogger(__name__)
@@ -23,6 +24,10 @@ PAUSED_KEY = "queue:paused"
LOOP_INTERVAL = 30 # 초
HOLIDAYS_REFRESH = 3600 # 1시간
PAUSED_TTL = 600 # 10분 (watcher 죽어도 자동 해제)
HEARTBEAT_KEY = "worker:task-watcher:heartbeat"
HEARTBEAT_TTL = 45 # LOOP_INTERVAL 30s < TTL 45s → 만료 전 갱신
_HB_STATS = WorkerStats()
async def watcher_loop():
@@ -46,6 +51,13 @@ async def watcher_loop():
else:
await redis.delete(PAUSED_KEY)
# heartbeat (LOOP_INTERVAL=30s < TTL 45s → 만료 전 갱신)
await redis.set(
HEARTBEAT_KEY,
build_payload("task-watcher", "watcher", mode, _HB_STATS, extra={"mode": mode}),
ex=HEARTBEAT_TTL,
)
if mode != last_mode:
logger.info("mode 전환: %s%s (paused=%s)", last_mode, mode, mode == "trading")
last_mode = mode

View File

@@ -0,0 +1,18 @@
# Plan-realtime-trade-alerts — trade-monitor
# NAS Redis (heartbeat)
REDIS_URL=redis://192.168.45.54:6379
# NAS stock 백엔드 (monitor-set / report)
NAS_BASE_URL=http://192.168.45.54:18500
WEBAI_API_KEY=
# KIS 자체 토큰 (ai_trade와 분리된 전용 app_key)
TM_KIS_APP_KEY=
TM_KIS_APP_SECRET=
TM_KIS_ACCOUNT=
TM_KIS_IS_VIRTUAL=0
# 루프 주기(초) / sell_climax 거래량 배수 임계
TM_LOOP_INTERVAL=60
TM_CLIMAX_VOL_MULT=3.0

View File

@@ -0,0 +1,166 @@
# trade-monitor 워커 — 구현 설계
> **2026-07-03 · web-ai 소유.** 실시간 매매 알람 파이프라인의 Windows-side 워커.
> **권위 계약(원본 스펙)**: web-backend repo `docs/superpowers/specs/2026-07-02-realtime-trade-alerts-design.md` §5(계약)·§6(조건).
> 이 문서는 그 계약을 Windows Docker 워커로 구현하기 위한 **구현 설계**(모듈 분해·조건 해석·배포)만 다룬다.
---
## 1. 역할 · 경계
`services/trade-monitor/` 는 형제 워커(`image-render`, `task-watcher`)와 동일한 관례를 따르는 **FastAPI + asyncio 루프** 워커다. WSL2 Docker(`services/docker-compose.yml`)에서 구동.
**책임**
1. 60초 루프로 NAS `monitor-set` 조회 → 세션 게이트.
2. 비-KRX(알파벳) 티커 skip.
3. KIS 실시간 현재가 + 일봉 OHLCV 조회 → TA 지표 계산.
4. 매수/매도 조건 평가 → 발화집합 F 구성.
5. `POST report`로 F 전체 전송(무상태 — dedup은 NAS 영속).
6. Redis heartbeat 발신(`worker:trade-monitor:heartbeat` EX45).
7. KIS 오류는 사이클/종목 단위 격리(다음 분 재시도).
**경계 밖(안 함)**: dedup 상태 보관, 텔레그램 전송(NAS 담당), KST 세션/휴장 캘린더 재구현(NAS가 `session` 판정), 주문 실행.
---
## 2. 모듈 분해
| 파일 | 책임 | 인터페이스(순수/부작용) |
|------|------|------|
| `main.py` | FastAPI app + lifespan. `monitor_loop` + `heartbeat_loop` 스폰, `/health` | 부작용(태스크 스폰) |
| `monitor.py` | 오케스트레이션 루프. monitor-set→게이트→종목순회→firing→report. 공유 `MonitorState` 갱신 | 부작용 |
| `nas_client.py` | `get_monitor_set()` / `post_report(as_of, firing)``X-WebAI-Key` + retry | 부작용(HTTP) |
| `kis_client.py` | KIS REST: `_issue_token()`(OAuth 자체 발급, 24h 캐시) + `get_quote()` + `get_daily_ohlcv()` + 0.5s throttle | 부작용(HTTP) |
| `indicators.py` | `sma`, `rsi`, `avg_volume`, `highest_high` | **순수** |
| `conditions.py` | `evaluate_buy(ctx, buy_params)` / `evaluate_sell(ctx, exit_params)``list[firing]` | **순수** |
| `config.py` | `Settings` — env 로드 | 순수 |
순수 모듈(`indicators`, `conditions`)에 조건 로직을 격리해 테이블 기반 단위 테스트로 검증 가능하게 한다. HTTP·시간·Redis는 경계 모듈에만.
---
## 3. 데이터 흐름 (monitor_loop, 60초)
```
매 사이클:
ms = nas.get_monitor_set() # §5.1
state.session = ms.session
if ms.session == "closed":
state.hb = "market_closed"; sleep; continue # KIS 호출 0
targets = filter_krx(ms.buy_targets, ms.sell_targets) # 알파벳 티커 skip
firing = []
for t in targets: # 종목 단위 try/except 격리
quote = kis.get_quote(t.ticker) # 현재가 + 당일 누적 거래량
daily = kis.get_daily_ohlcv(t.ticker, 250) # MA200·52주 고점용
ctx = build_ctx(t, quote, daily)
if t is buy_target: firing += evaluate_buy(ctx, ms.buy_params)
if t is sell_target: firing += evaluate_sell(ctx, ms.exit_params)
if firing: state.last_alert_at = now
nas.post_report(as_of=now_kst_iso, firing=firing) # §5.2 — 빈 배열도 전송(edge clear 위해)
state.hb = "market_open"
stats.jobs_done += 1
```
`heartbeat_loop`(별도 15초 태스크)이 `state`를 읽어 §5.4 페이로드 발신.
---
## 4. 조건 로직 해석 (§6)
지표는 **일봉 시계열**(최신 last, 오름차순) + **실시간 현재가**(`price`) 기준. 데이터 부족(예: MA200용 200봉 미만)이면 해당 조건은 **미발화**(graceful skip). 각 firing은 `{ticker, kind, condition, price, detail}`.
### 매수 (`buy_targets`, `buy_params={rsi_oversold, breakout_vol_mult, pullback_pct}`)
| condition | 발화 규칙(해석) | detail |
|-----------|----------------|--------|
| `buy_ma20_pullback` | `ma20>ma50>ma200`(정배열) **AND** 최근 3봉 최저가가 `ma20*(1+pullback_pct)` 이하로 접근 **AND** `price>ma20`(반등 복귀) | `ma20, ma50, ma200, recent_low` |
| `buy_breakout` | `price > 직전 20봉 최고가`(당일 제외) **AND** `today_volume > breakout_vol_mult × avg_volume(20)` | `prior_high_20, vol_mult, avg_vol_20` |
| `buy_rsi_bounce` | RSI(14) 시계열에서 `min(rsi[-3:]) < rsi_oversold` **AND** `rsi[-1] > rsi_oversold` **AND** `rsi[-1] > rsi[-2]`(반등). 사이클마다 재계산·무상태 | `rsi, rsi_prev, rsi_oversold` |
종합: 각 조건 독립 발화(신뢰도 가중합은 NAS/텔레그램 단계 책임 아님 — 워커는 조건 발화만).
### 매도 (`sell_targets`, `exit_params={stop_pct, take_pct, trailing_pct}`, target에 `avg_price, qty, holding_high`)
| condition | 발화 규칙 | detail |
|-----------|----------|--------|
| `sell_stop_loss` | `(price-avg)/avg ≤ -stop_pct` | `avg_price, pnl_pct, stop_pct` |
| `sell_take_profit` | `(price-avg)/avg ≥ take_pct` | `avg_price, pnl_pct, take_pct` |
| `sell_trailing_stop` | `price ≤ holding_high × (1-trailing_pct)` (기본 0.10) | `holding_high, trailing_pct, drawdown_pct` |
| `sell_ma_break` | `price < ma50` (추가 `price<ma200`이면 detail.severity="high") | `ma50, ma200, severity` |
| `sell_climax` | **holdings_intel 정합**: `today_volume ≥ climax_vol_x × avg_volume(20)` **AND** `price < day_high × climax_close_pct`(윗꼬리) | `vol_mult, day_high, climax_close_pct` |
`climax_vol_x`(기본 3.0)·`climax_close_pct`(기본 0.97)는 monitor-set `exit_params`에서 읽음(BE 중앙화, main ed17193). 없으면 env `TM_CLIMAX_VOL_MULT` fallback. `day_high`는 KIS quote `stck_hgpr`(당일 세션 누적 고가).
---
## 5. KIS 클라이언트 (자체 토큰)
- `_issue_token()`: `POST {base}/oauth2/tokenP {grant_type, appkey, appsecret}``access_token`(만료 24h). 메모리 캐시, 만료 10분 전 재발급. **ai_trade와 분리된 `TM_KIS_APP_KEY/SECRET`** 사용(같은 app_key 공유 시 토큰 상호 무효화 + EGW00201).
- `get_quote(ticker)`: `inquire-price`(FHKST01010100) → `stck_prpr`(현재가), `acml_vol`(당일 누적 거래량), `stck_oprc`(당일 시가).
- `get_daily_ohlcv(ticker, days=250)`: `inquire-daily-itemchartprice`(FHKST03010100) — ai_trade `kis_client.py` 로직 복제, 오름차순.
- throttle 0.5s(초당 2회) + `_throttle_lock` 직렬화 + 429/timeout 지수 backoff(ai_trade 패턴 재사용).
> ⚠️ **운영 함정**: ai_trade와 KIS를 동시 호출하면 전용 app_key라도 KIS 계정 전체 rate limit을 공유할 수 있음. 별도 app_key로 무효화는 회피되나, 운영 시 동시 부하 모니터링 필요(Phase 7 백로그 연계).
---
## 6. heartbeat (§5.4)
`_shared.heartbeat.heartbeat_loop(redis, "trade-monitor", "trader", stats, interval=15, ttl=45, state_fn=...)`.
- `state_fn``MonitorState`를 읽어 `state ∈ {market_open, market_closed, idle}` + `extra={"last_alert_at": ...}` 반환.
- **디커플링 이유**: 루프 60초 > TTL 45초 → 인라인 발신 시 만료 갭. 15초 독립 태스크로 해소(형제 워커와 동일 구조). §5.4 필수 필드(name/kind/state/ts/last_alert_at) 충족, `jobs_done/jobs_failed`는 형제 워커처럼 superset 유지.
- 초기 상태 `idle`(첫 monitor-set 조회 전).
---
## 7. 설정 (env) — `TM_` 접두사로 ai_trade와 분리
| env | 기본값 | 용도 |
|-----|--------|------|
| `NAS_BASE_URL` | `http://192.168.45.54:18500` | stock 백엔드 |
| `WEBAI_API_KEY` | (필수) | `X-WebAI-Key` |
| `REDIS_URL` | `redis://192.168.45.54:6379` | heartbeat |
| `TM_KIS_APP_KEY` / `TM_KIS_APP_SECRET` | (필수) | KIS 자체 토큰 |
| `TM_KIS_ACCOUNT` | (필수) | KIS 계좌 |
| `TM_KIS_IS_VIRTUAL` | `0` | 실전/모의 |
| `TM_LOOP_INTERVAL` | `60` | 루프 주기(초) |
| `TM_CLIMAX_VOL_MULT` | `3.0` | sell_climax 임계 |
---
## 8. 에러 처리
- **monitor-set 실패**: 사이클 skip(report 안 함), heartbeat=`idle`, 다음 분 재시도.
- **KIS 종목 실패**: 해당 종목만 skip(로그 warning), 나머지 종목 계속.
- **report 실패**: 로그 error, 다음 사이클 신선 firing 재전송(무상태라 손실 허용).
- 루프 최상위 `try/except` — 어떤 예외도 루프를 죽이지 않음(task-watcher 패턴).
---
## 9. 테스트 전략 (pytest, 시스템 Python)
| 파일 | 검증 |
|------|------|
| `test_indicators.py` | sma/rsi/avg_volume/highest_high 수치(알려진 시계열), 데이터 부족 시 None |
| `test_conditions.py` | 8개 조건 테이블 기반(발화/미발화 경계), detail 필드 |
| `test_nas_client.py` | respx — monitor-set 파싱, report 페이로드, X-WebAI-Key 헤더, retry |
| `test_kis_client.py` | respx — 토큰 발급/캐시, quote/daily 파싱, throttle |
| `test_monitor.py` | 루프 1회(mock): closed skip, 비-KRX skip, firing 조립, last_alert_at 갱신, 종목 실패 격리 |
---
## 10. 배포
- `services/trade-monitor/Dockerfile`: task-watcher 관례 복제 — `COPY _shared /app/_shared` **필수**(빌드 컨텍스트 `.` 에서), `COPY trade-monitor/. /app/`, `PYTHONPATH=/app`, uvicorn `:8000`.
- `services/docker-compose.yml`: `trade-monitor` 서비스 추가, 포트 **18715**(image-render 18714 다음), `TZ=Asia/Seoul`, KIS/WEBAI/REDIS env, healthcheck `/health`.
- `services/.env`(비커밋): `TM_KIS_*`, `WEBAI_API_KEY` 실값. `.env.example`에 키만 기재.
---
## 11. 미해결 플래그 / 후속
1. **sell_climax** — ✅ 2026-07-03 holdings_intel 정합 완료(`price < day_high × climax_close_pct` + `exit_params` 파라미터화). BE 회신 기준.
2. **KIS 지표 필드 실검증** — quote의 `acml_vol`/`stck_oprc`, daily TR 응답 필드는 첫 운영 raw 캡처로 대조.
3. **`buy_ma20_pullback`·`buy_rsi_bounce` 해석** — "current candle series" 문구를 일봉 시계열로 해석. 첫 운영 4주 IC 검증 시 재조정 가능.
4. **KIS rate limit 공존** — ai_trade와 동시 부하. 전용 app_key로 토큰 무효화는 회피, 초당 호출 총량은 운영 모니터링.
5. **after 세션 시간외 시세**`inquire-price`가 시간외 단일가를 반영하는지 첫 운영 대조.

View File

@@ -0,0 +1,19 @@
FROM python:3.12-slim-bookworm
ENV PYTHONUNBUFFERED=1
WORKDIR /app
RUN apt-get update && apt-get install -y --no-install-recommends \
ca-certificates tzdata \
&& rm -rf /var/lib/apt/lists/*
COPY trade-monitor/requirements.txt /app/
RUN pip install --no-cache-dir --timeout 600 --retries 5 -r requirements.txt
# 공통 heartbeat 모듈 (services/_shared) — main.py가 from _shared.heartbeat import
COPY _shared /app/_shared
COPY trade-monitor/. /app/
ENV PYTHONPATH=/app
EXPOSE 8000
CMD ["python", "-m", "uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000", "--workers", "1"]

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,99 @@
"""§6 조건 로직 (순수). ctx + params → firing 리스트."""
from __future__ import annotations
from indicators import sma, rsi_series, highest_high
def _fire(ctx: dict, kind: str, condition: str, price: float, detail: dict) -> dict:
return {
"ticker": ctx["ticker"], "kind": kind,
"condition": condition, "price": price, "detail": detail,
}
def evaluate_buy(ctx: dict, params: dict) -> list[dict]:
price = ctx["price"]
closes, highs, lows, vols = ctx["closes"], ctx["highs"], ctx["lows"], ctx["volumes"]
rsi_os = params.get("rsi_oversold", 30)
vol_mult = params.get("breakout_vol_mult", 1.5)
pullback = params.get("pullback_pct", 0.02)
firing: list[dict] = []
# buy_ma20_pullback — 정배열 + ma20 근접 저가 + 반등 복귀
ma20, ma50, ma200 = sma(closes, 20), sma(closes, 50), sma(closes, 200)
if ma20 and ma50 and ma200 and ma20 > ma50 > ma200 and len(lows) >= 3:
recent_low = min(lows[-3:])
if recent_low <= ma20 * (1 + pullback) and price > ma20:
firing.append(_fire(ctx, "buy", "buy_ma20_pullback", price, {
"ma20": round(ma20, 1), "ma50": round(ma50, 1),
"ma200": round(ma200, 1), "recent_low": recent_low,
}))
# buy_breakout — 직전 20봉 고점 돌파 + 거래량 배수
prior_high20 = highest_high(highs, 20)
avg_vol20 = sma(vols, 20)
if prior_high20 and avg_vol20 and price > prior_high20 \
and ctx["today_volume"] > vol_mult * avg_vol20:
firing.append(_fire(ctx, "buy", "buy_breakout", price, {
"prior_high_20": prior_high20,
"vol_mult": round(ctx["today_volume"] / avg_vol20, 2),
"avg_vol_20": round(avg_vol20, 0),
}))
# buy_rsi_bounce — RSI 과매도 후 반등 (무상태 재계산)
rs = rsi_series(closes, 14)
if len(rs) >= 3 and min(rs[-3:]) < rsi_os and rs[-1] > rsi_os and rs[-1] > rs[-2]:
firing.append(_fire(ctx, "buy", "buy_rsi_bounce", price, {
"rsi": round(rs[-1], 1), "rsi_prev": round(rs[-2], 1),
"rsi_oversold": rsi_os,
}))
return firing
def evaluate_sell(ctx: dict, params: dict) -> list[dict]:
price = ctx["price"]
avg = ctx.get("avg_price")
hh = ctx.get("holding_high")
closes, vols = ctx["closes"], ctx["volumes"]
stop = params.get("stop_pct", 0.08)
take = params.get("take_pct", 0.25)
trail = params.get("trailing_pct", 0.10)
firing: list[dict] = []
if avg:
pnl = (price - avg) / avg
if pnl <= -stop:
firing.append(_fire(ctx, "sell", "sell_stop_loss", price, {
"avg_price": avg, "pnl_pct": round(pnl, 4), "stop_pct": stop}))
if pnl >= take:
firing.append(_fire(ctx, "sell", "sell_take_profit", price, {
"avg_price": avg, "pnl_pct": round(pnl, 4), "take_pct": take}))
if hh and price <= hh * (1 - trail):
firing.append(_fire(ctx, "sell", "sell_trailing_stop", price, {
"holding_high": hh, "trailing_pct": trail,
"drawdown_pct": round((price - hh) / hh, 4)}))
ma50, ma200 = sma(closes, 50), sma(closes, 200)
if ma50 and price < ma50:
severity = "high" if (ma200 and price < ma200) else "normal"
firing.append(_fire(ctx, "sell", "sell_ma_break", price, {
"ma50": round(ma50, 1),
"ma200": round(ma200, 1) if ma200 else None,
"severity": severity}))
# sell_climax — holdings_intel 정합(stock/app/holdings_intel.py:109-118):
# 거래량 ≥ 20일평균 × climax_vol_x AND 종가 < 당일고가 × climax_close_pct (윗꼬리)
# 실시간이므로 day_high = 당일 세션 누적 고가(최신 1분봉 고가 아님).
climax_vol_x = params.get("climax_vol_x", ctx.get("climax_vol_mult", 3.0))
climax_close_pct = params.get("climax_close_pct", 0.97)
avg_vol20 = sma(vols, 20)
day_high = ctx.get("day_high")
if avg_vol20 and day_high and ctx["today_volume"] >= climax_vol_x * avg_vol20 \
and price < day_high * climax_close_pct:
firing.append(_fire(ctx, "sell", "sell_climax", price, {
"vol_mult": round(ctx["today_volume"] / avg_vol20, 2),
"day_high": day_high, "climax_close_pct": climax_close_pct}))
return firing

View File

@@ -0,0 +1,32 @@
"""Settings — 환경변수 로드. TM_ 접두사로 ai_trade와 분리."""
from __future__ import annotations
import os
from dataclasses import dataclass
@dataclass
class Settings:
nas_base_url: str
webai_api_key: str
redis_url: str
kis_app_key: str
kis_app_secret: str
kis_account: str
kis_is_virtual: bool
loop_interval: int
climax_vol_mult: float
def load_settings() -> Settings:
return Settings(
nas_base_url=os.getenv("NAS_BASE_URL", "http://192.168.45.54:18500"),
webai_api_key=os.getenv("WEBAI_API_KEY", ""),
redis_url=os.getenv("REDIS_URL", "redis://192.168.45.54:6379"),
kis_app_key=os.getenv("TM_KIS_APP_KEY", ""),
kis_app_secret=os.getenv("TM_KIS_APP_SECRET", ""),
kis_account=os.getenv("TM_KIS_ACCOUNT", ""),
kis_is_virtual=os.getenv("TM_KIS_IS_VIRTUAL", "0") == "1",
loop_interval=int(os.getenv("TM_LOOP_INTERVAL", "60")),
climax_vol_mult=float(os.getenv("TM_CLIMAX_VOL_MULT", "3.0")),
)

View File

@@ -0,0 +1,5 @@
"""services/ 루트를 sys.path에 추가 — from _shared.heartbeat import 가능하게."""
import sys
from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))

View File

@@ -0,0 +1,38 @@
"""순수 TA 지표 — sma / rsi_series / highest_high."""
from __future__ import annotations
def sma(values: list[float], period: int) -> float | None:
if period <= 0 or len(values) < period:
return None
return sum(values[-period:]) / period
def highest_high(highs: list[float], period: int) -> float | None:
if period <= 0 or len(highs) < period:
return None
return max(highs[-period:])
def rsi_series(closes: list[float], period: int = 14) -> list[float]:
"""Wilder RSI. 반환 리스트는 closes[period:]에 1:1 정렬. 부족하면 []."""
if len(closes) <= period:
return []
deltas = [closes[i] - closes[i - 1] for i in range(1, len(closes))]
gains = [d if d > 0 else 0.0 for d in deltas]
losses = [-d if d < 0 else 0.0 for d in deltas]
def _rsi(ag: float, al: float) -> float:
if al == 0:
return 100.0
rs = ag / al
return 100.0 - 100.0 / (1.0 + rs)
avg_gain = sum(gains[:period]) / period
avg_loss = sum(losses[:period]) / period
out = [_rsi(avg_gain, avg_loss)]
for i in range(period, len(deltas)):
avg_gain = (avg_gain * (period - 1) + gains[i]) / period
avg_loss = (avg_loss * (period - 1) + losses[i]) / period
out.append(_rsi(avg_gain, avg_loss))
return out

View File

@@ -0,0 +1,124 @@
"""KIS REST client — 자체 OAuth 토큰(TM_KIS_*) + quote + 일봉 + throttle."""
from __future__ import annotations
import asyncio
import logging
import time
from datetime import datetime, timedelta
from zoneinfo import ZoneInfo
import httpx
logger = logging.getLogger(__name__)
KST = ZoneInfo("Asia/Seoul")
_MAX_ATTEMPTS = 3
_THROTTLE_INTERVAL = 0.5 # 초당 2회
_TOKEN_MARGIN = 600 # 만료 10분 전 재발급
class KISClient:
def __init__(self, app_key, app_secret, account, is_virtual, timeout: float = 10.0):
self._app_key = app_key
self._app_secret = app_secret
self._account = account
self._base_url = (
"https://openapivts.koreainvestment.com:29443" if is_virtual
else "https://openapi.koreainvestment.com:9443"
)
self._client = httpx.AsyncClient(timeout=timeout)
self._token: str | None = None
self._token_exp: float = 0.0
self._last_throttle_at = 0.0
self._throttle_lock = asyncio.Lock()
self._token_lock = asyncio.Lock()
async def close(self) -> None:
await self._client.aclose()
async def _issue_token(self) -> str:
async with self._token_lock:
now = time.time()
if self._token and now < self._token_exp - _TOKEN_MARGIN:
return self._token
r = await self._client.post(
f"{self._base_url}/oauth2/tokenP",
json={"grant_type": "client_credentials",
"appkey": self._app_key, "appsecret": self._app_secret},
)
r.raise_for_status()
data = r.json()
self._token = data["access_token"]
self._token_exp = now + int(data.get("expires_in", 86400))
return self._token
async def _throttle(self) -> None:
async with self._throttle_lock:
elapsed = time.monotonic() - self._last_throttle_at
if elapsed < _THROTTLE_INTERVAL:
await asyncio.sleep(_THROTTLE_INTERVAL - elapsed)
self._last_throttle_at = time.monotonic()
async def _request(self, method: str, path: str, tr_id: str, **kwargs) -> dict:
token = await self._issue_token()
headers = {
"authorization": f"Bearer {token}",
"appkey": self._app_key, "appsecret": self._app_secret,
"tr_id": tr_id, "custtype": "P",
}
url = f"{self._base_url}{path}"
for attempt in range(_MAX_ATTEMPTS):
await self._throttle()
try:
resp = await self._client.request(method, url, headers=headers, **kwargs)
if resp.status_code == 429 and attempt < _MAX_ATTEMPTS - 1:
await asyncio.sleep(2 ** attempt)
continue
resp.raise_for_status()
return resp.json()
except httpx.TimeoutException:
if attempt < _MAX_ATTEMPTS - 1:
await asyncio.sleep(2 ** attempt)
continue
raise
raise RuntimeError("retry exhausted")
async def get_quote(self, ticker: str) -> dict:
raw = await self._request(
"GET", "/uapi/domestic-stock/v1/quotations/inquire-price",
tr_id="FHKST01010100",
params={"FID_COND_MRKT_DIV_CODE": "J", "FID_INPUT_ISCD": ticker},
)
o = raw.get("output", {})
return {
"price": int(o["stck_prpr"]),
"day_open": int(o["stck_oprc"]),
"day_high": int(o["stck_hgpr"]),
"today_volume": int(o["acml_vol"]),
"as_of": datetime.now(KST).isoformat(),
}
async def get_daily_ohlcv(self, ticker: str, days: int = 250) -> list[dict]:
today = datetime.now(KST).strftime("%Y%m%d")
start = (datetime.now(KST) - timedelta(days=days * 2)).strftime("%Y%m%d")
raw = await self._request(
"GET", "/uapi/domestic-stock/v1/quotations/inquire-daily-itemchartprice",
tr_id="FHKST03010100",
params={"FID_COND_MRKT_DIV_CODE": "J", "FID_INPUT_ISCD": ticker,
"FID_INPUT_DATE_1": start, "FID_INPUT_DATE_2": today,
"FID_PERIOD_DIV_CODE": "D", "FID_ORG_ADJ_PRC": "1"},
)
bars = []
for row in raw.get("output2", []):
try:
d = row["stck_bsop_date"]
bars.append({
"datetime": f"{d[:4]}-{d[4:6]}-{d[6:]}",
"open": int(row["stck_oprc"]), "high": int(row["stck_hgpr"]),
"low": int(row["stck_lwpr"]), "close": int(row["stck_clpr"]),
"volume": int(row["acml_vol"]),
})
except (KeyError, ValueError):
continue
bars.reverse()
return bars[-days:]

View File

@@ -0,0 +1,62 @@
"""trade-monitor FastAPI entry — lifespan(monitor_loop + heartbeat_loop) + /health."""
from __future__ import annotations
import asyncio
import logging
from contextlib import asynccontextmanager
import redis.asyncio as aioredis
from fastapi import FastAPI
import monitor
from config import load_settings
from kis_client import KISClient
from nas_client import NASClient
from _shared.heartbeat import heartbeat_loop, WorkerStats
logging.basicConfig(level=logging.INFO,
format="%(asctime)s %(name)s %(levelname)s %(message)s")
logger = logging.getLogger(__name__)
HEARTBEAT_INTERVAL = 15 # 60초 루프 > TTL 45초 → 독립 15초 발신으로 만료갭 해소
HEARTBEAT_TTL = 45
@asynccontextmanager
async def lifespan(app: FastAPI):
settings = load_settings()
nas = NASClient(settings.nas_base_url, settings.webai_api_key)
kis = KISClient(settings.kis_app_key, settings.kis_app_secret,
settings.kis_account, settings.kis_is_virtual)
state = monitor.MonitorState()
stats = WorkerStats()
redis = aioredis.from_url(settings.redis_url, decode_responses=False)
mon_task = asyncio.create_task(
monitor.monitor_loop(nas, kis, state, stats, settings))
hb_task = asyncio.create_task(heartbeat_loop(
redis, "trade-monitor", "trader", stats,
interval=HEARTBEAT_INTERVAL, ttl=HEARTBEAT_TTL,
state_fn=monitor.make_state_fn(state)))
logger.info("trade-monitor lifespan 시작")
try:
yield
finally:
for t in (mon_task, hb_task):
t.cancel()
try:
await t
except asyncio.CancelledError:
pass
await kis.close()
await nas.close()
await redis.aclose()
logger.info("trade-monitor lifespan 종료")
app = FastAPI(lifespan=lifespan)
@app.get("/health")
def health():
return {"ok": True, "service": "trade-monitor"}

View File

@@ -0,0 +1,114 @@
"""오케스트레이션 — monitor-set 조회 → 조건 평가 → report + heartbeat state."""
from __future__ import annotations
import asyncio
import logging
from datetime import datetime
from zoneinfo import ZoneInfo
from conditions import evaluate_buy, evaluate_sell
logger = logging.getLogger(__name__)
KST = ZoneInfo("Asia/Seoul")
class MonitorState:
"""monitor_loop가 갱신, heartbeat state_fn이 읽는 공유 상태."""
def __init__(self):
self.session_state = "idle" # market_open | market_closed | idle
self.last_alert_at: str | None = None
def filter_krx(targets: list[dict]) -> list[dict]:
"""6자리 숫자 티커(KRX)만. 알파벳 티커 skip."""
out = []
for t in targets:
tk = str(t.get("ticker", ""))
if tk.isdigit() and len(tk) == 6:
out.append(t)
return out
async def _build_ctx(kis, target: dict, settings) -> dict:
ticker = target["ticker"]
quote = await kis.get_quote(ticker)
daily = await kis.get_daily_ohlcv(ticker, 250)
return {
"ticker": ticker, "name": target.get("name", ""),
"price": quote["price"], "day_open": quote["day_open"],
"day_high": quote["day_high"],
"today_volume": quote["today_volume"],
"closes": [b["close"] for b in daily],
"highs": [b["high"] for b in daily],
"lows": [b["low"] for b in daily],
"volumes": [b["volume"] for b in daily],
"avg_price": target.get("avg_price"),
"qty": target.get("qty"),
"holding_high": target.get("holding_high"),
"climax_vol_mult": settings.climax_vol_mult,
}
async def run_cycle(nas, kis, state, stats, settings) -> None:
try:
ms = await nas.get_monitor_set()
except Exception:
logger.exception("monitor-set 조회 실패")
state.session_state = "idle"
stats.jobs_failed += 1
return
session = ms.get("session", "closed")
if session == "closed":
state.session_state = "market_closed"
return
buy_targets = filter_krx(ms.get("buy_targets", []))
sell_targets = filter_krx(ms.get("sell_targets", []))
buy_params = ms.get("buy_params", {})
exit_params = ms.get("exit_params", {})
firing: list[dict] = []
for t in buy_targets:
try:
firing += evaluate_buy(await _build_ctx(kis, t, settings), buy_params)
except Exception:
logger.exception("buy 평가 실패 %s", t.get("ticker"))
for t in sell_targets:
try:
firing += evaluate_sell(await _build_ctx(kis, t, settings), exit_params)
except Exception:
logger.exception("sell 평가 실패 %s", t.get("ticker"))
as_of = datetime.now(KST).isoformat(timespec="seconds")
if firing:
state.last_alert_at = as_of
logger.info("firing %d개: %s", len(firing),
[f"{f['ticker']}:{f['condition']}" for f in firing])
try:
await nas.post_report(as_of, firing) # 빈 배열도 전송(edge clear)
except Exception:
logger.exception("report 전송 실패")
state.session_state = "market_open"
stats.jobs_done += 1
stats.last_job_at = as_of
async def monitor_loop(nas, kis, state, stats, settings) -> None:
logger.info("trade-monitor loop 시작 interval=%ds", settings.loop_interval)
while True:
try:
await run_cycle(nas, kis, state, stats, settings)
except asyncio.CancelledError:
logger.info("monitor_loop cancelled")
raise
except Exception:
logger.exception("monitor_loop iteration 실패")
await asyncio.sleep(settings.loop_interval)
def make_state_fn(state):
async def state_fn(redis, stats):
return state.session_state, {"last_alert_at": state.last_alert_at}
return state_fn

View File

@@ -0,0 +1,48 @@
"""NAS stock 백엔드 trade-alert 계약 — X-WebAI-Key + retry."""
from __future__ import annotations
import asyncio
import logging
import httpx
logger = logging.getLogger(__name__)
_MAX_ATTEMPTS = 3
_RETRY_STATUSES = {429, 500, 502, 503, 504}
class NASClient:
def __init__(self, base_url: str, api_key: str, timeout: float = 10.0):
self._base_url = base_url.rstrip("/")
self._api_key = api_key
self._client = httpx.AsyncClient(timeout=timeout)
async def close(self) -> None:
await self._client.aclose()
async def get_monitor_set(self) -> dict:
return await self._request("GET", "/api/webai/trade-alert/monitor-set")
async def post_report(self, as_of: str, firing: list[dict]) -> dict:
return await self._request(
"POST", "/api/webai/trade-alert/report",
json={"as_of": as_of, "firing": firing})
async def _request(self, method: str, path: str, **kwargs) -> dict:
url = f"{self._base_url}{path}"
headers = {"X-WebAI-Key": self._api_key}
for attempt in range(_MAX_ATTEMPTS):
try:
resp = await self._client.request(method, url, headers=headers, **kwargs)
if resp.status_code in _RETRY_STATUSES and attempt < _MAX_ATTEMPTS - 1:
await asyncio.sleep(2 ** attempt)
continue
resp.raise_for_status()
return resp.json()
except httpx.TimeoutException:
if attempt < _MAX_ATTEMPTS - 1:
await asyncio.sleep(2 ** attempt)
continue
raise
raise RuntimeError("retry exhausted")

View File

@@ -0,0 +1,2 @@
[pytest]
asyncio_mode = auto

View File

@@ -0,0 +1,7 @@
fastapi==0.115.6
uvicorn[standard]==0.34.0
redis>=5.0
httpx>=0.27
pytest>=8.0
pytest-asyncio>=0.24
respx>=0.21

View File

View File

@@ -0,0 +1,66 @@
"""evaluate_buy — 3개 매수 조건 경계."""
from conditions import evaluate_buy
BUY_PARAMS = {"rsi_oversold": 30, "breakout_vol_mult": 1.5, "pullback_pct": 0.02}
def _ctx(**over):
base = dict(
ticker="005930", name="삼성전자", price=100.0, day_open=99.0,
today_volume=1000.0, closes=[], highs=[], lows=[], volumes=[],
avg_price=None, qty=None, holding_high=None, climax_vol_mult=3.0,
)
base.update(over)
return base
def _conditions(firing):
return {f["condition"] for f in firing}
def test_ma20_pullback_fires():
# 정배열(ma20>ma50>ma200), 최근 저가가 ma20 근처, price가 ma20 위로 반등
closes = [90.0] * 200 + [100.0] * 20 # ma20=100, ma50/ma200 낮음 → 정배열
lows = [90.0] * 217 + [100.5, 100.4, 100.3] # 최근 3봉 저가 ~ma20*(1.02)=102 이하
ctx = _ctx(price=101.0, closes=closes, highs=closes, lows=lows,
volumes=[1.0] * len(closes))
assert "buy_ma20_pullback" in _conditions(evaluate_buy(ctx, BUY_PARAMS))
def test_ma20_pullback_skips_when_not_aligned():
closes = [100.0] * 200 + [90.0] * 20 # 역배열
ctx = _ctx(price=91.0, closes=closes, highs=closes, lows=closes,
volumes=[1.0] * len(closes))
assert "buy_ma20_pullback" not in _conditions(evaluate_buy(ctx, BUY_PARAMS))
def test_breakout_fires():
closes = [50.0] * 25
highs = [60.0] * 25 # 직전 20봉 최고 60
vols = [100.0] * 25 # avg20=100
ctx = _ctx(price=61.0, today_volume=200.0, closes=closes, highs=highs,
lows=closes, volumes=vols) # 61>60, 200>1.5*100
assert "buy_breakout" in _conditions(evaluate_buy(ctx, BUY_PARAMS))
def test_breakout_skips_on_low_volume():
highs = [60.0] * 25
ctx = _ctx(price=61.0, today_volume=120.0, closes=[50.0] * 25, highs=highs,
lows=[50.0] * 25, volumes=[100.0] * 25) # 120 < 1.5*100=150
assert "buy_breakout" not in _conditions(evaluate_buy(ctx, BUY_PARAMS))
def test_rsi_bounce_fires():
# 14봉 급락으로 RSI<30 찍고 5봉 반등하여 30 위로 복귀
closes = [100.0]
for _ in range(14):
closes.append(closes[-1] * 0.97) # 하락 → RSI 저하
for _ in range(5):
closes.append(closes[-1] * 1.05) # 반등 → RSI 30 위로
ctx = _ctx(price=closes[-1], closes=closes, highs=closes, lows=closes,
volumes=[1.0] * len(closes))
assert "buy_rsi_bounce" in _conditions(evaluate_buy(ctx, BUY_PARAMS))
def test_empty_series_no_fire():
assert evaluate_buy(_ctx(), BUY_PARAMS) == []

View File

@@ -0,0 +1,89 @@
"""evaluate_sell — 5개 매도 조건 경계."""
from conditions import evaluate_sell
EXIT = {"stop_pct": 0.08, "take_pct": 0.25, "trailing_pct": 0.10,
"climax_vol_x": 3.0, "climax_close_pct": 0.97}
def _ctx(**over):
base = dict(
ticker="000660", name="SK하이닉스", price=100.0, day_open=100.0,
day_high=100.0, today_volume=100.0, closes=[100.0] * 60,
highs=[100.0] * 60, lows=[100.0] * 60, volumes=[100.0] * 60,
avg_price=100.0, qty=10, holding_high=100.0, climax_vol_mult=3.0,
)
base.update(over)
return base
def _c(firing):
return {f["condition"] for f in firing}
def test_stop_loss_fires():
ctx = _ctx(price=90.0, avg_price=100.0) # -10% <= -8%
assert "sell_stop_loss" in _c(evaluate_sell(ctx, EXIT))
def test_stop_loss_skips_above_threshold():
ctx = _ctx(price=95.0, avg_price=100.0) # -5% > -8%
assert "sell_stop_loss" not in _c(evaluate_sell(ctx, EXIT))
def test_take_profit_fires():
ctx = _ctx(price=130.0, avg_price=100.0) # +30% >= 25%
assert "sell_take_profit" in _c(evaluate_sell(ctx, EXIT))
def test_trailing_stop_fires():
ctx = _ctx(price=89.0, holding_high=100.0) # 89 <= 100*0.9=90
assert "sell_trailing_stop" in _c(evaluate_sell(ctx, EXIT))
def test_ma_break_severity_high():
# price가 ma50/ma200 아래 → severity high (ma200 계산 위해 200봉 필요)
closes = [200.0] * 200
ctx = _ctx(price=100.0, closes=closes, avg_price=100.0, holding_high=100.0)
firing = evaluate_sell(ctx, EXIT)
mb = [f for f in firing if f["condition"] == "sell_ma_break"]
assert mb and mb[0]["detail"]["severity"] == "high"
def test_climax_fires():
# holdings_intel 정합: 거래량 3배 이상 + 종가 < 당일고가×0.97 (윗꼬리)
ctx = _ctx(price=96.0, day_high=100.0, today_volume=400.0,
volumes=[100.0] * 60) # 400>=3*100, 96 < 100*0.97=97
assert "sell_climax" in _c(evaluate_sell(ctx, EXIT))
def test_climax_skips_when_not_reversal():
# 종가가 당일고가의 97% 이상 → 윗꼬리 아님
ctx = _ctx(price=99.0, day_high=100.0, today_volume=400.0,
volumes=[100.0] * 60) # 99 >= 100*0.97=97 → 반전 아님
assert "sell_climax" not in _c(evaluate_sell(ctx, EXIT))
def test_climax_uses_exit_params_vol_x():
# exit_params.climax_vol_x=5.0 → 400 < 5*100=500 → 미발화
exit5 = {**EXIT, "climax_vol_x": 5.0}
ctx = _ctx(price=96.0, day_high=100.0, today_volume=400.0,
volumes=[100.0] * 60)
assert "sell_climax" not in _c(evaluate_sell(ctx, exit5))
def test_climax_uses_exit_params_close_pct():
# climax_close_pct=0.90 → 임계 90, price=95 → 95<90? No → 미발화
exit90 = {**EXIT, "climax_close_pct": 0.90}
ctx = _ctx(price=95.0, day_high=100.0, today_volume=400.0,
volumes=[100.0] * 60)
assert "sell_climax" not in _c(evaluate_sell(ctx, exit90))
# 기본 0.97이면 95 < 97 → 발화
assert "sell_climax" in _c(evaluate_sell(ctx, EXIT))
def test_no_avg_no_pnl_conditions():
# avg_price None(보유정보 없음) → stop/take 미발화
ctx = _ctx(price=50.0, avg_price=None, holding_high=None,
closes=[100.0] * 60)
conds = _c(evaluate_sell(ctx, EXIT))
assert "sell_stop_loss" not in conds and "sell_take_profit" not in conds

View File

@@ -0,0 +1,27 @@
"""Settings env 로드 — 기본값 + override."""
from config import load_settings
def test_defaults(monkeypatch):
for k in ("NAS_BASE_URL", "WEBAI_API_KEY", "REDIS_URL", "TM_KIS_APP_KEY",
"TM_KIS_APP_SECRET", "TM_KIS_ACCOUNT", "TM_KIS_IS_VIRTUAL",
"TM_LOOP_INTERVAL", "TM_CLIMAX_VOL_MULT"):
monkeypatch.delenv(k, raising=False)
s = load_settings()
assert s.nas_base_url == "http://192.168.45.54:18500"
assert s.redis_url == "redis://192.168.45.54:6379"
assert s.kis_is_virtual is False
assert s.loop_interval == 60
assert s.climax_vol_mult == 3.0
def test_override(monkeypatch):
monkeypatch.setenv("TM_KIS_IS_VIRTUAL", "1")
monkeypatch.setenv("TM_LOOP_INTERVAL", "30")
monkeypatch.setenv("TM_CLIMAX_VOL_MULT", "2.5")
monkeypatch.setenv("WEBAI_API_KEY", "secret")
s = load_settings()
assert s.kis_is_virtual is True
assert s.loop_interval == 30
assert s.climax_vol_mult == 2.5
assert s.webai_api_key == "secret"

View File

@@ -0,0 +1,6 @@
"""/health — 라우트 핸들러 직접 검증."""
from main import health
def test_health():
assert health() == {"ok": True, "service": "trade-monitor"}

View File

@@ -0,0 +1,39 @@
"""indicators — 순수 수치 검증."""
from indicators import sma, rsi_series, highest_high
def test_sma_basic():
assert sma([1, 2, 3, 4, 5], 5) == 3.0
assert sma([1, 2, 3, 4, 5], 2) == 4.5
def test_sma_insufficient():
assert sma([1, 2], 5) is None
assert sma([], 3) is None
def test_highest_high():
assert highest_high([1, 9, 3, 4], 3) == 9
assert highest_high([1, 2, 3], 3) == 3
assert highest_high([1, 2], 3) is None
def test_rsi_all_gains_is_100():
# 단조 증가 → 손실 0 → RSI 100
closes = [float(i) for i in range(1, 20)]
rs = rsi_series(closes, 14)
assert rs, "series should not be empty"
assert rs[-1] == 100.0
def test_rsi_insufficient():
assert rsi_series([1, 2, 3], 14) == []
def test_rsi_known_range():
# 등락 섞인 시계열 → RSI는 0~100 사이
closes = [10, 11, 10.5, 11.5, 11, 12, 11.8, 12.5, 12, 13,
12.7, 13.2, 12.9, 13.5, 13.1, 13.8]
rs = rsi_series(closes, 14)
assert len(rs) == len(closes) - 14
assert all(0.0 <= v <= 100.0 for v in rs)

View File

@@ -0,0 +1,56 @@
"""KISClient — 토큰 발급/캐시 + quote/daily 파싱 (respx)."""
import httpx
import respx
from kis_client import KISClient
BASE = "https://openapi.koreainvestment.com:9443"
def _client():
return KISClient("APPKEY", "APPSECRET", "12345678-01", is_virtual=False)
@respx.mock
async def test_issue_token_cached():
route = respx.post(f"{BASE}/oauth2/tokenP").mock(
return_value=httpx.Response(200, json={"access_token": "TKN", "expires_in": 86400}))
c = _client()
t1 = await c._issue_token()
t2 = await c._issue_token()
assert t1 == "TKN" and t2 == "TKN"
assert route.call_count == 1 # 캐시 → 1회만 발급
await c.close()
@respx.mock
async def test_get_quote_parses():
respx.post(f"{BASE}/oauth2/tokenP").mock(
return_value=httpx.Response(200, json={"access_token": "TKN", "expires_in": 86400}))
respx.get(f"{BASE}/uapi/domestic-stock/v1/quotations/inquire-price").mock(
return_value=httpx.Response(200, json={"output": {
"stck_prpr": "71500", "stck_oprc": "71000", "stck_hgpr": "72000",
"acml_vol": "1234567"}}))
c = _client()
q = await c.get_quote("005930")
assert q["price"] == 71500 and q["day_open"] == 71000 and q["today_volume"] == 1234567
assert q["day_high"] == 72000
await c.close()
@respx.mock
async def test_get_daily_ascending():
respx.post(f"{BASE}/oauth2/tokenP").mock(
return_value=httpx.Response(200, json={"access_token": "TKN", "expires_in": 86400}))
# KIS는 내림차순 반환 → 오름차순으로 뒤집혀야 함
respx.get(f"{BASE}/uapi/domestic-stock/v1/quotations/inquire-daily-itemchartprice").mock(
return_value=httpx.Response(200, json={"output2": [
{"stck_bsop_date": "20260702", "stck_oprc": "100", "stck_hgpr": "110",
"stck_lwpr": "90", "stck_clpr": "105", "acml_vol": "5"},
{"stck_bsop_date": "20260701", "stck_oprc": "95", "stck_hgpr": "102",
"stck_lwpr": "94", "stck_clpr": "100", "acml_vol": "4"}]}))
c = _client()
bars = await c.get_daily_ohlcv("005930", days=250)
assert bars[0]["datetime"] == "2026-07-01"
assert bars[-1]["close"] == 105
await c.close()

View File

@@ -0,0 +1,95 @@
"""monitor.run_cycle — 게이트/필터/조립/격리."""
from monitor import MonitorState, filter_krx, run_cycle
from config import load_settings
from _shared.heartbeat import WorkerStats
def test_filter_krx_keeps_only_numeric6():
targets = [{"ticker": "005930"}, {"ticker": "AAPL"}, {"ticker": "00660"},
{"ticker": "000660"}, {"ticker": "0059301"}]
kept = {t["ticker"] for t in filter_krx(targets)}
assert kept == {"005930", "000660"}
class _FakeNAS:
def __init__(self, ms):
self._ms = ms
self.reported = None
async def get_monitor_set(self):
return self._ms
async def post_report(self, as_of, firing):
self.reported = {"as_of": as_of, "firing": firing}
return {"new_alerts": len(firing), "cleared": 0}
class _FakeKIS:
def __init__(self, price=100, fail_on=None):
self._price = price
self._fail_on = fail_on or set()
async def get_quote(self, ticker):
if ticker in self._fail_on:
raise RuntimeError("KIS down")
return {"price": self._price, "day_open": 99, "day_high": 100,
"today_volume": 1000, "as_of": "x"}
async def get_daily_ohlcv(self, ticker, days=250):
# 정배열 + 저가 근접 → ma20_pullback 발화 유도
return [{"open": 90, "high": 90, "low": 90, "close": 90, "volume": 1}] * 200 \
+ [{"open": 100, "high": 100, "low": 100, "close": 100, "volume": 1}] * 20
async def test_closed_session_skips_kis():
nas = _FakeNAS({"session": "closed"})
state, stats = MonitorState(), WorkerStats()
await run_cycle(nas, _FakeKIS(), state, stats, load_settings())
assert state.session_state == "market_closed"
assert nas.reported is None # report도 안 함
async def test_non_krx_skipped_and_report_sent():
nas = _FakeNAS({"session": "regular",
"buy_targets": [{"ticker": "AAPL", "name": "Apple"}],
"sell_targets": [], "buy_params": {}, "exit_params": {}})
state, stats = MonitorState(), WorkerStats()
await run_cycle(nas, _FakeKIS(), state, stats, load_settings())
assert state.session_state == "market_open"
assert nas.reported is not None
assert nas.reported["firing"] == [] # 알파벳 티커 skip → 빈 발화
async def test_firing_assembled_and_last_alert_set():
nas = _FakeNAS({"session": "regular",
"buy_targets": [{"ticker": "005930", "name": "삼성전자"}],
"sell_targets": [], "buy_params": {"pullback_pct": 0.02},
"exit_params": {}})
state, stats = MonitorState(), WorkerStats()
await run_cycle(nas, _FakeKIS(price=101), state, stats, load_settings())
conds = {f["condition"] for f in nas.reported["firing"]}
assert "buy_ma20_pullback" in conds
assert state.last_alert_at is not None
async def test_per_ticker_failure_isolated():
nas = _FakeNAS({"session": "regular",
"buy_targets": [{"ticker": "005930"}, {"ticker": "000660"}],
"sell_targets": [], "buy_params": {}, "exit_params": {}})
state, stats = MonitorState(), WorkerStats()
# 005930은 실패, 000660은 성공 → 루프가 죽지 않고 report 전송
await run_cycle(nas, _FakeKIS(fail_on={"005930"}), state, stats, load_settings())
assert nas.reported is not None
assert state.session_state == "market_open"
async def test_monitor_set_failure_sets_idle():
class _BadNAS(_FakeNAS):
async def get_monitor_set(self):
raise RuntimeError("NAS down")
nas = _BadNAS({})
state, stats = MonitorState(), WorkerStats()
await run_cycle(nas, _FakeKIS(), state, stats, load_settings())
assert state.session_state == "idle"
assert nas.reported is None

View File

@@ -0,0 +1,39 @@
"""NASClient — monitor-set/report + X-WebAI-Key (respx)."""
import json as _json
import httpx
import respx
from nas_client import NASClient
BASE = "http://nas.test"
@respx.mock
async def test_get_monitor_set_sends_key():
route = respx.get(f"{BASE}/api/webai/trade-alert/monitor-set").mock(
return_value=httpx.Response(200, json={"session": "regular", "buy_targets": []}))
c = NASClient(BASE, "KEY")
ms = await c.get_monitor_set()
assert ms["session"] == "regular"
assert route.calls.last.request.headers["X-WebAI-Key"] == "KEY"
await c.close()
@respx.mock
async def test_post_report_payload():
captured = {}
def _resp(request):
captured.update(_json.loads(request.content))
return httpx.Response(200, json={"new_alerts": 1, "cleared": 0})
respx.post(f"{BASE}/api/webai/trade-alert/report").mock(side_effect=_resp)
c = NASClient(BASE, "KEY")
firing = [{"ticker": "005930", "kind": "buy", "condition": "buy_breakout",
"price": 71500, "detail": {}}]
out = await c.post_report("2026-07-02T09:01:00+09:00", firing)
assert out["new_alerts"] == 1
assert captured["as_of"] == "2026-07-02T09:01:00+09:00"
assert captured["firing"] == firing
await c.close()

View File

@@ -3,11 +3,14 @@ from __future__ import annotations
import asyncio
import logging
import os
from contextlib import asynccontextmanager
import redis.asyncio as aioredis
from fastapi import FastAPI
import worker
from _shared.heartbeat import heartbeat_loop
logging.basicConfig(level=logging.INFO, format="%(asctime)s %(name)s %(levelname)s %(message)s")
logger = logging.getLogger(__name__)
@@ -16,15 +19,19 @@ logger = logging.getLogger(__name__)
@asynccontextmanager
async def lifespan(app: FastAPI):
worker_task = asyncio.create_task(worker.worker_loop())
hb_redis = aioredis.from_url(os.getenv("REDIS_URL", "redis://192.168.45.54:6379"), decode_responses=False)
hb_task = asyncio.create_task(heartbeat_loop(hb_redis, "video-render", "render", worker.stats))
logger.info("video-render lifespan 시작")
try:
yield
finally:
worker_task.cancel()
for t in (worker_task, hb_task):
t.cancel()
try:
await worker_task
await t
except asyncio.CancelledError:
pass
await hb_redis.aclose()
logger.info("video-render lifespan 종료")

View File

@@ -94,3 +94,25 @@ async def test_poll_once_returns_false_on_timeout(monkeypatch):
assert handled is False
fake_queue.ack.assert_not_awaited()
fake_queue.fail.assert_not_awaited()
# ----- heartbeat stats 카운터 -----
class _OneJobQueue:
def __init__(self): self.acked = False
async def dequeue(self, timeout=5):
if self.acked: return None
return ({"job_type": "sora_generation", "task_id": "t1", "params": {}}, b"raw")
async def ack(self, raw): self.acked = True
async def fail(self, raw, payload): pass
@pytest.mark.asyncio
async def test_poll_once_increments_jobs_done(monkeypatch):
worker.stats.jobs_done = 0
monkeypatch.setattr(worker, "run_sora_generation", lambda task_id, params: None)
handled = await worker.poll_once(_OneJobQueue())
assert handled is True
assert worker.stats.jobs_done == 1
assert worker.stats.busy is False
assert worker.stats.last_job_at is not None

View File

@@ -19,6 +19,7 @@ from providers.veo import run_veo_generation
from providers.kling import run_kling_generation
from providers.seedance import run_seedance_generation
from _shared.reliable_queue import ReliableQueue
from _shared.heartbeat import WorkerStats, utc_now_iso
logger = logging.getLogger(__name__)
@@ -26,6 +27,8 @@ REDIS_URL = os.getenv("REDIS_URL", "redis://192.168.45.54:6379")
QUEUE_KEY = "queue:video-render"
PAUSED_KEY = "queue:paused"
stats = WorkerStats()
# string names so `unittest.mock.patch` on `worker.<name>` is correctly intercepted
_DISPATCH_TABLE = {
"sora_generation": "run_sora_generation",
@@ -60,14 +63,21 @@ async def poll_once(queue: ReliableQueue) -> bool:
if result is None:
return False
payload, raw = result
stats.busy = True
try:
await asyncio.to_thread(_dispatch, payload)
except Exception:
logger.exception("dispatch unhandled exception task_id=%s",
payload.get("task_id"))
await queue.fail(raw, payload)
stats.jobs_failed += 1
stats.last_job_at = utc_now_iso()
stats.busy = False
return True
await queue.ack(raw)
stats.jobs_done += 1
stats.last_job_at = utc_now_iso()
stats.busy = False
return True