# hoody-pipe Subskill ## Overview ### What is Hoody Pipe? Hoody Pipe is a real-time data streaming service that enables direct data transfer between clients using named pipe paths. Unlike traditional file storage or message queues, Pipe streams data directly from sender to receiver with zero server-side storage — data flows through the pipe in real-time. ### When to Use Hoody Pipe | Use Case | Description | |----------|-------------| | **File Transfer** | Send files between machines without intermediate storage | | **Log Streaming** | Stream application logs in real-time to monitoring tools | | **Data Pipeline** | Connect data producers to consumers with backpressure | | **Remote Execution** | Pipe command output between distributed processes | | **Browser Upload** | Use the web interface for manual file/text submission | ### How Pipe Fits Hoody Philosophy Hoody Pipe embodies the Hoody service principles: - **Zero Configuration**: Named paths are created on-demand — no pre-registration required - **Direct Streaming**: Data flows sender→receiver without disk I/O or message brokers - **Protocol Flexibility**: Supports file uploads, text streams, and binary data - **Container Isolation**: Each container has independent pipe namespaces - **Authentication Built-in**: All pipe operations require valid Hoody authentication - **Automatic Routing**: Access via standardized Hoody proxy URLs ### Architecture ``` ┌─────────┐ POST/PUT ┌─────────────┐ GET (blocks) ┌─────────┐ │ Sender │ ──────────────────▶│ Hoody Pipe │─────────────────────▶│Receiver │ │ │ /pipe/{path} │ (stream) │ /pipe/{path} │ │ └─────────┘ └─────────────┘ └─────────┘ │ ▼ No storage — direct relay ``` ### Key Characteristics - **Blocking Receivers**: GET requests block until a sender connects - **Streaming Lifecycle**: Connection ends when sender completes transmission - **Path-Based Routing**: Any string can be a pipe path — paths are virtual - **Multiple Receivers**: Multiple GET requests to the same path can receive the same stream - **CORS Support**: Browser-based access via OPTIONS preflight --- ## Common Workflows ### Workflow 1: Basic File Transfer Between Containers Transfer a file from one container to another using named pipes. **Step 1: Start receiver on target container** The receiver blocks until data arrives: ``` hoody pipe receive mydata -c target-container-123 -o json ``` Expected behavior: Command blocks, waiting for sender connection. **Step 2: Send file from source container** In a separate terminal, send the file: ``` hoody pipe send mydata -c source-container-456 --file ./report.csv ``` **Step 3: Verify transfer** The receiver completes when the sender finishes streaming. Check the output: ``` { "status": "received", "path": "mydata", "bytes": 1024, "contentType": "text/csv" } ``` ### Workflow 2: Stream Command Output Between Processes Pipe real-time output from one process to another. **Step 1: Start receiver listening for logs** ``` hoody pipe receive app-logs -c monitor-container-789 ``` **Step 2: Stream application output** ``` hoody pipe send app-logs -c app-container-012 --stream ``` The receiver displays data as it arrives in real-time. ### Workflow 3: Browser-Based File Upload Use the Pipe web interface for manual uploads from a browser. **Step 1: Get the web interface URL** ``` hoody pipe web -c my-container-abc ``` This returns the URL for the Pipe web interface: ``` https://project123-containerabc-pipe-service1.node.containers.hoody.icu/api/v1/pipe ``` **Step 2: Open in browser** Navigate to the returned URL. The interface provides: - File upload form - Text input area - Path specification field **Step 3: Specify target path and upload** Enter a path name (e.g., `uploads/report`) and submit your file or text. ### Workflow 4: No-JavaScript Upload Form For restricted environments where JavaScript is disabled. **Step 1: Get the noscript form URL** ``` hoody pipe web --noscript -c my-container-abc ``` **Step 2: Use with query parameters** The noscript form accepts query parameters: ``` /api/v1/pipe/noscript?path=my-upload-path ``` This pre-fills the target path in the form. ### Workflow 5: Check Service Health Verify the Pipe service is operational. ``` hoody pipe health -c my-container-abc ``` Expected response: ``` { "status": "healthy", "service": "pipe", "version": "1.0.0", "uptime": 86400, "timestamp": "2025-01-15T10:30:00Z", "container": "my-container-abc", "region": "us-east-1", "node": "node1", "checks": { "memory": "ok", "connections": "ok" } } ``` ### Workflow 6: Get Usage Help Retrieve built-in usage instructions with server-specific examples. ``` hoody pipe help -c my-container-abc ``` Returns plain text with curl examples using the server's own URL — examples are ready to copy and run. ### Workflow 7: Text Message Transfer Send a text message through a pipe. **Step 1: Start receiver** ``` hoody pipe receive messages -c receiver-container-111 ``` **Step 2: Send text message** ``` hoody pipe send messages -c sender-container-222 --data "Hello from sender" ``` **Step 3: Verify receipt** Receiver output: ``` { "status": "received", "path": "messages", "contentType": "text/plain", "data": "Hello from sender" } ``` ### Workflow 8: Verify Transfer Completion Check that a pipe transfer completed successfully. **Step 1: Initiate transfer with verbose output** ``` hoody pipe send mydata -c source-container --file ./data.bin --verbose ``` **Step 2: Check sender confirmation** ``` { "status": "sent", "path": "mydata", "bytes": 4096, "duration": "1.2s" } ``` --- ## Advanced Operations ### Complex Workflow 1: Multi-Stage Data Pipeline Chain multiple pipe operations for ETL-style processing. **Stage 1: Raw data ingestion** ``` # Terminal 1: Start processor listening for raw data hoody pipe receive raw-data -c processor-container-001 ``` ``` # Terminal 2: Send raw data from source hoody pipe send raw-data -c source-container-002 --file ./raw-input.json ``` **Stage 2: Processed output** ``` # Terminal 3: Start consumer listening for processed data hoody pipe receive processed-data -c consumer-container-003 ``` ``` # Terminal 4: After processing, send results hoody pipe send processed-data -c processor-container-001 --file ./output.json ``` **Stage 3: Verify pipeline completion** ``` hoody pipe health -c processor-container-001 ``` ### Complex Workflow 2: Broadcast to Multiple Receivers Send the same data to multiple receivers simultaneously. **Step 1: Start all receivers on the same path** ``` # Receiver 1 hoody pipe receive broadcast-data -c receiver-1 # Receiver 2 hoody pipe receive broadcast-data -c receiver-2 # Receiver 3 hoody pipe receive broadcast-data -c receiver-3 ``` **Step 2: Send data once** ``` hoody pipe send broadcast-data -c sender-container --file ./announcement.txt ``` All three receivers receive the same data stream. ### Error Recovery Patterns #### Pattern 1: Sender Timeout Recovery If a sender disconnects before a receiver connects: ``` # Receiver waits with timeout hoody pipe receive mydata -c target-container --timeout 30 ``` If timeout occurs: ``` { "status": "timeout", "path": "mydata", "message": "No sender connected within 30 seconds" } ``` **Recovery**: Retry the receive operation or check sender availability. #### Pattern 2: Receiver Disconnect During Transfer If receiver disconnects mid-stream: ``` # Sender receives notification hoody pipe send mydata -c source-container --file ./large-file.bin ``` ``` { "status": "partial", "path": "mydata", "bytesSent": 1024, "bytesTotal": 4096, "error": "Receiver disconnected" } ``` **Recovery**: Restart receiver and resend from beginning. #### Pattern 3: Authentication Failure ``` hoody pipe send mydata -c my-container --file ./data.txt ``` ``` { "status": "error", "code": 401, "message": "Authentication required" } ``` **Recovery**: Authenticate first: ``` hoody auth login hoody pipe send mydata -c my-container --file ./data.txt ``` ### Performance Considerations #### Large File Transfers For files > 100MB: ``` # Use streaming mode for large files hoody pipe send large-data -c source-container --file ./huge-dataset.csv --stream ``` **Best practices:** - Use `--stream` flag for files > 100MB - Ensure stable network connection - Monitor transfer with `--verbose` flag #### Concurrent Pipe Operations Multiple pipes can operate simultaneously on different paths: ``` # These run concurrently without interference hoody pipe send path-a -c container-1 --file ./a.txt & hoody pipe send path-b -c container-1 --file ./b.txt & hoody pipe send path-c -c container-1 --file ./c.txt & ``` #### Connection Pooling For high-throughput scenarios: ``` # Reuse authentication session hoody auth login --session mysession # All subsequent commands use cached session hoody pipe send data1 -c container --file ./f1.txt --profile mysession hoody pipe send data2 -c container --file ./f2.txt --profile mysession ``` ### Advanced: Custom Content Types Send data with specific content types: ``` # Send JSON with explicit content type hoody pipe send config -c app-container --file ./config.json --content-type application/json # Send binary data hoody pipe send binary-data -c target-container --file ./image.png --content-type image/png ``` --- ## Quick Reference ### Essential Commands | Command | Description | |---------|-------------| | `hoody pipe send -c --file ` | Send file to pipe | | `hoody pipe send -c --data ""` | Send text to pipe | | `hoody pipe receive -c ` | Receive data from pipe (blocks) | | `hoody pipe health -c ` | Check service health | | `hoody pipe help -c ` | Get usage instructions | | `hoody pipe web -c ` | Get web interface URL | | `hoody pipe web --noscript -c ` | Get noscript form URL | ### Common Flags | Flag | Description | |------|-------------| | `-c ` | Target container (required) | | `--file ` | File to send | | `--data ""` | Text data to send | | `--stream` | Enable streaming mode for large files | | `--timeout ` | Receive timeout (default: infinite) | | `--verbose` | Show detailed transfer information | | `--content-type ` | Specify content MIME type | | `-o json` | Output in JSON format | ### Pipe Path Naming Paths are virtual and created on-demand: ``` mydata # Simple path project/logs/app # Hierarchical path user-123/uploads # User-specific path temp/transfer-abc123 # Temporary path ``` ### Typical Response Formats **Successful send:** ``` { "status": "sent", "path": "mydata", "bytes": 1024, "duration": "0.5s" } ``` **Successful receive:** ``` { "status": "received", "path": "mydata", "bytes": 1024, "contentType": "application/octet-stream" } ``` **Health check:** ``` { "status": "healthy", "service": "pipe", "version": "1.0.0", "uptime": 86400, "timestamp": "2025-01-15T10:30:00Z" } ``` **Error:** ``` { "status": "error", "code": 404, "message": "Container not found" } ``` ### Proxy URL Pattern Access Pipe service via Hoody proxy: ``` https://{projectId}-{containerId}-pipe-{serviceId}.{node}.containers.hoody.icu/api/v1/pipe/{path} ``` ### Authentication ``` # Login first hoody auth login # Or use token hoody pipe send mydata -c container --file ./data.txt --token # Or use environment variable export HOODY_TOKEN= hoody pipe send mydata -c container --file ./data.txt ```