How to: Monitor your App

Effective monitoring is crucial for understanding if your Apps are functioning correctly. This guide explains how Syftbox captures logs and how you can use them to monitor your Apps.

How Syftbox captures logs

When you run a App in Syftbox, all output is automatically captured and saved:

• Standard Output (stdout): All print() statements, normal program output
• Standard Error (stderr): All error messages and warnings
• Shell Command Output: Output from any commands in shell scripts like echo

No special configuration is needed - Syftbox automatically redirects and saves these streams to log files.

Debugging & monitoring your local App

1. View logs through the CLI SyftBox client

syftbox client

Whenever you run the client, you will see the live logs coming from all the Apps that are running on your system. This helps to give you a high-level overview of what is happening.

2. Check log files directly

In many cases when you are developing or debugging a specific App, the high level view becomes cluttered. In this case, you can look at the logs of a App independently, which are stored under:

Syftbox/apps/<your_app>/logs

These files contain the complete execution history and can be analyzed after runs complete. To stream the logs in real-time, you can use the tail command:

tail ./Syftbox/apps/<your_app>/logs

3. Textual UI Interface (Beta)

For a more interactive monitoring experience:

syftbox tui

This provides a terminal-based UI to browse Apps and view their status and logs.

Debugging and monitoring your App for others

You can only monitor App executions on your local machine. You cannot directly observe if your App is working correctly for other users or in other environments. There are different methods that can be used to empower you with the ability to proactively solve the problems your users are facing:

1. Automate error reporting to a public location

Modify your App to automatically report errors to a publicly accessible location:

import traceback
import os
import json
from datetime import datetime, UTC
from syftbox.lib import Client, SyftPermission

def report_error(error_msg, error_trace):
    client = Client.load()
    error_log_folder = client.app_data("error_logs")

    # Ensure folder exists
    os.makedirs(error_log_folder, exist_ok=True)

    # Create error report
    error_file = error_log_folder / f"error_{datetime.now(UTC).strftime('%Y%m%d_%H%M%S')}.json"
    error_data = {
        "timestamp": datetime.now(UTC).isoformat(),
        "error_message": error_msg,
        "error_trace": error_trace
    }

    # Write error data
    with open(error_file, "w") as f:
        json.dump(error_data, f, indent=2)

    # Set public read permissions
    permission = SyftPermission.mine_with_public_read(email=client.email)
    permission.ensure(error_log_folder)

    print(f"Error report saved to {error_file}")

try:
    # Your App code here
    main()
except Exception as e:
    error_msg = str(e)
    error_trace = traceback.format_exc()
    report_error(error_msg, error_trace)
    raise

2. Create a publicly readable status log

def log_execution_status(status, message):
    client = Client.load()
    status_folder = client.app_data("status_logs")

    # Ensure folder exists with public read permissions
    os.makedirs(status_folder, exist_ok=True)

    # Write status entry
    status_file = status_folder / "execution_status.jsonl"
    status_entry = {
        "timestamp": datetime.now(UTC).isoformat(),
        "status": status,
        "message": message
    }

    with open(status_file, "a") as f:
        f.write(json.dumps(status_entry) + "\n")

    # Set permissions
    permission = SyftPermission.mine_with_public_read(email=client.email)
    permission.ensure(status_folder)

    print(f"Status '{status}' logged: {message}")

Usage example:

def main():
    try:
        log_execution_status("started", "App execution started")

        # Your App logic here

        log_execution_status("completed", "App completed successfully")
    except Exception as e:
        log_execution_status("failed", f"App failed: {str(e)}")
        raise

Best Practices

1. Appropriate logging: Since all stdout is captured, use print statements strategically to log important events and not have it overwhelmed by unnecessary information.

2. Include timestamps: Always include timestamps in your custom logs to track when events occurred.

3. Set appropriate permissions: Make sure logs that need to be accessed by others have public read permissions using SyftPermission.mine_with_public_read().

4. Don't log sensitive information: Remember that logs might be accessible to others, so avoid logging sensitive data.

5. Different environments: Test your App in different environments to ensure it works as expected. It is expected that your users might have different operating systems or software versions.

We are working on improving your ability to monitor and identify issues - in the meantime, hope this allows you to quickly identify and resolve any issues that arise.