Refines Platform Stability and Expands Core Capabilities#8
Conversation
…d network hardening Introduce WIFI_STATUS_ONLINE state that distinguishes "has IP" from "verified internet access" via periodic TCP probe to 8.8.8.8:53. Add cross-core toast notification system (spinlock-guarded queue, drawn via display flush hook). Harden TCP driver with per-connection spinlocks for thread-safe rx_count/pending/state access between Core 0 and Core 1. Other changes: - HTTP/TCP connect and read timeout enforcement on Core 1 - SNTP retry logic (3 attempts with 5s backoff) - Native app stack moved from 8KB static SRAM to 64KB PSRAM (umm_malloc) - fs API extended with mkdir/delete/rename/isDir - picocalc.network.http.isComplete() and picocalc.ui.toast() Lua bindings - wifi_get_ip/wifi_get_ssid use spinlock-protected copies for Core 0 reads
…nd process management Add display_stats, display_diff, and get_pixel MCP tools for framebuffer inspection. Fix Unicorn emulator nested callback handling via SVC #254 sentinel and restart loop. Add kill_simulators/list_simulators for process lifecycle management. Fix sdcard_fsize_handle, sdcard_stat, and add FS trampoline stubs (mkdir, delete, rename, is_dir). Update WiFi sim to use WIFI_STATUS_ONLINE. Increase emulator code region to 8MB and stack to 256KB.
Eliminates cross-core PSRAM heap contention between Lua (Core 0) and Mongoose (Core 1). Core 1 now uses a 128KB first-fit allocator carved from the end of the PSRAM region, requiring no cross-core locking. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Flush XIP cache after core1_alloc_init (Core 1 cache coherency) - Add overflow check in core1_calloc (count * size) - Add bounds check in core1_free via core1_owns() - Fix misleading comment on block_t.next field Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace spinlock-guarded ring buffer with C11 atomic head/tail indices. Single producer (Core 0) / single consumer (Core 1) pattern needs no lock — just acquire/release memory ordering. Eliminates IRQ masking on the IPC hot path. State protection (s_ip, s_ssid) uses a separate s_state_lock spinlock. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Prevents indefinite device freeze when SD card degrades. Each block already has a 500ms per-block timeout, but with no outer limit a card that intermittently responds could stall Core 0 for minutes. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
native_loader.c (200ms) and usb_msc.c (500ms) previously waited indefinitely for Core 1 to acknowledge pause. If Core 1 was stuck in a long TLS operation, Core 0 would spin forever. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
wifi_get_status() is called from Core 0 while s_status and s_internet_ok are written by Core 1. C11 _Atomic ensures no torn reads on RP2350's per-core caches. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
LCD uses PIO0 SPI, not hardware SPI1. There is no bus sharing with CYW43 and display_spi_lock()/display_spi_unlock() don't exist. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Defense-in-depth: USB ISR now acquires g_sdcard_mutex before disk_read/disk_write. Previously relied solely on Core 1 being paused. The recursive mutex is uncontended (no other holder during MSC mode) so this adds safety with zero overhead. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
40 MHz caused card resets (R1=0x3f) during heavy WiFi/PSRAM activity. Adds sdcard_ensure_ready() with remount recovery, fopen retry on DISK_ERR, corruption log rate-limiting during sustained failures, CMD23 failure recovery in multi-block writes, and sd_set_slow_mode() for callers that write while CYW43 radio may be active. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…ndler Replace volatile with _Atomic and proper acquire/release memory ordering for all Core 0 <-> Core 1 shared variables (g_core1_pause, ring buffer indices, g_native_audio_callback). Adds __dmb() barriers at critical handshake points. Also enhances HardFault handler to show running app name and uptime on both UART and LCD, packs PSP flag + uptime into scratch[7] for crash log persistence, adds launcher_get_app_uptime_ms(), Core 1 pause timeout in launcher_apply_clock, ELF segment layout guards, stack overflow detection improvement, and pauseBackground/resumeBackground Lua APIs. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Display: fillVLine (stride-based, no Bresenham), drawTexturedColumn (16.16 fixed-point texture sampling), fillVLineGradient (RGB565 channel interpolation). Exposed to Lua and native SDK. Toast: replace icon-based system with style-based severity levels (info/success/warning/error) with color-coded backgrounds. Add bg_color parameter to ui_widget_toast. Toast now auto-drawn during display flush. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Pre-reads entire firmware into PSRAM so the flash write loop (SRAM- resident) never calls flash-resident code (SD, FatFS, display). Writes sector 0 last so boot code stays intact if interrupted. Uses direct ARM AIRCR reset instead of flash-resident watchdog_reboot(). Moves OTA flag from scratch[0] to scratch[1] to avoid boot-loop counter conflict. Renames update files before flash writes begin. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Adds wifi_hw_disconnected() for callers that need to wait for CYW43 radio power-down before voltage-sensitive ops. Spinlock-protects credential copy in wifi_connect(). Exposes ensureReady/setSlowMode (fs), isHwDisconnected (network), to Lua. HTTP: adds dmb + timeout warning in http_free, callback error logging, bumps read buffer cap to 128KB, setReadBufferSize returns bool. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
WAV parser: bounds check on malformed chunk sizes. Fileplayer: early return if WAV buffer alloc fails. Simulator: ELF image size bounds check, setReadBufferSize bool return. Mongoose: TLS recv buffer safety. SDK: terminal.h gains line numbers, scrollbar, render bounds, and word wrap APIs. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Prevents large binaries from being accidentally committed: doom1.wad, main.elf, inspiration.jpg, tic-80/, screenshot.png Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace vendored miniz files with a git submodule from richgel999/miniz. The PicOS-specific allocator config (miniz_picos.h) lives at third_party/miniz_picos.h since submodule contents shouldn't be modified. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add picocalc.zip Lua API and g_api.zip C API for extracting ZIP archives from SD card. Uses miniz (decompression only) with allocations redirected to PSRAM via umm_malloc. - lua_bridge_zip.c: picocalc.zip.extract() and picocalc.zip.list() - main.c: s_zip_impl wrappers using sdcard_read_file + miniz - os.h: picocalc_zip_t struct added to PicoCalcAPI - simulator CMakeLists: miniz sources with decompression-only flags Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add sdcard_delete_recursive() for removing directories with all contents, exposed as picocalc.fs.deleteRecursive() in Lua. Needed for app uninstall and cleanup operations. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add pio_psram_available, pio_psram_size, and xip_cache_hit_rate fields to the Lua memory info table for performance tuning. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Fix cull condition (> 0 instead of < 0) and light Z component (-0.802f instead of 0.802f) for proper triangle visibility and lighting. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Extend app metadata with fields for app store categorization: - category: games, tools, system, demos, emulators, network - repo: source repository reference - min_firmware: minimum compatible firmware version Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Redesign launcher UI with category-based navigation: - 3x2 grid home view with procedural category icons - Category-filtered app list with icon thumbnails - Three view states: home, category, all apps - PNG/BMP icon loading with colored square fallback Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add cgb_lut[64] for cached RGB555->RGB565 palette conversion - Remove blocking spin-wait delays that prevented watchdog feeding - Move input polling inside emulation loop - Refresh LUT mid-game for dynamic palette changes Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add periodic sys->poll() calls in DG_SleepMs and throttled polling in DG_GetTicksMs to prevent watchdog timeout during WAD loading. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Rewrite 3D test to render a detailed PicoCalc device including body, screen, d-pad, buttons, and speaker grille with filled triangle rendering and color. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- editor: UI and functionality improvements - systest: expanded system test coverage - updater: enhanced OTA update flow - picoforge: use font name string instead of magic index Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Implement proper triangle fill (sorted vertex scanline) - Add stubs: textured column, vline, vline gradient - Add sdcard_ensure_ready and sd_set_slow_mode stubs - Make g_native_audio_callback atomic for thread safety - Add wifi_hw_disconnected stub Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
This pull request makes broad platform stability improvements (SD/OTA, networking, inter-core coordination), adds several new core OS APIs (ZIP, filesystem ops, display primitives, toast UI), and expands simulator diagnostics and process control.
Changes:
- Harden SD card read/write, add recovery paths, add SD “slow mode”, and rework OTA update to pre-read firmware into PSRAM with an SRAM-resident flash writer.
- Convert multiple cross-core/shared state paths to C11 atomics, add pause/resume handshakes, and introduce a dedicated Core 1 allocator pool.
- Add ZIP extraction/listing (miniz), toast notifications, new display primitives, expanded system metrics, and simulator tooling/diagnostics.
Reviewed changes
Copilot reviewed 82 out of 83 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
| tools/picos_mcp.py | Adds simulator PID tracking + list/kill utilities and display diagnostics MCP tools. |
| third_party/mongoose/mongoose.c | Minor TLS recv buffer handling tweak (space variable). |
| third_party/miniz_picos.h | Configures miniz allocation to use PSRAM allocator (umm). |
| third_party/fatfs/port/diskio_spi.c | SD SPI robustness: timeouts, recovery, staging buffer, slow mode, retry logic. |
| src/usb/usb_msc.c | Uses atomics for Core 1 pause flags; adds SD mutex defense-in-depth in MSC callbacks. |
| src/os/ui.c | Treats WIFI_STATUS_ONLINE as connected for header icon/color. |
| src/os/ui_widgets.h | Extends toast widget signature to accept background color. |
| src/os/ui_widgets.c | Implements toast background color parameter. |
| src/os/toast.h | Introduces system-wide toast API (queue + styles). |
| src/os/toast.c | Implements toast queue + draw routine with spinlock protection. |
| src/os/system_menu.c | Distinguishes CONNECTED vs ONLINE in WiFi status UI and toggle behavior. |
| src/os/ota_update.h | Moves OTA scratch flag to scratch[1]; reserves scratch[0] for boot-loop counter. |
| src/os/ota_update.c | Reworks OTA flow: pre-read firmware to PSRAM, SRAM-resident flash write + safe sector ordering. |
| src/os/os.h | Expands API structs (display primitives, fs ops, http changes, zip API); adds atomic callback type. |
| src/os/native_loader.c | Enlarges native stack (PSRAM), improves pause handshake, adds ELF layout guards, atomic audio callback clear. |
| src/os/lua_psram_alloc.h | Exposes Core 1 dedicated pool accessors. |
| src/os/lua_psram_alloc.c | Reserves Core 1 PSRAM pool on RP2350; adds pool accessors. |
| src/os/lua_bridge.c | Registers ZIP Lua bridge module. |
| src/os/lua_bridge_zip.h | Declares ZIP Lua bridge init. |
| src/os/lua_bridge_zip.c | Adds Lua ZIP list/extract using miniz with PSRAM allocations. |
| src/os/lua_bridge_ui.c | Adds UI API for system toasts + optional toast bg color in drawToast. |
| src/os/lua_bridge_sys.c | Adds sys.pauseBackground/resumeBackground; expands sys.getMemInfo metrics. |
| src/os/lua_bridge_network.c | Adds wifi.hasInternet, network.isHwDisconnected; improves Lua HTTP callback error logging; increases max read chunk. |
| src/os/lua_bridge_fs.c | Adds fs.deleteRecursive, fs.ensureReady, fs.setSlowMode. |
| src/os/lua_bridge_display.c | Composites toasts on flush; adds new display primitives to Lua API. |
| src/os/lua_bridge_3d.c | Fixes lighting/backface cull orientation logic for 3D demo rendering. |
| src/os/launcher.h | Adds app uptime accessor for crash context. |
| src/os/launcher_types.h | Adds app category and icon fields. |
| src/os/core1_alloc.h | Introduces Core 1 dedicated allocator API. |
| src/os/core1_alloc.c | Implements simple first-fit allocator for Core 1 PSRAM pool. |
| src/main.c | Adds crash context (app + uptime), toast compositing flush wrapper, ZIP API, Core 1 allocator init + cache clean, OTA fallback trigger. |
| src/hardware.h | Drops SD SPI baud to 25MHz to reduce errors during WiFi/PSRAM activity. |
| src/drivers/wifi.h | Adds internet verification + hw-disconnect completion APIs; updates docs. |
| src/drivers/wifi.c | Atomics for status/IPC queue, connectivity checks, SNTP retries, toasts on failures; timeout enforcement hooks. |
| src/drivers/video_player.cpp | Treats WIFI_STATUS_ONLINE as connected when disconnecting WiFi for clock boost. |
| src/drivers/tcp.h | Adds spinlock + connect timeout fields and timeout enforcement API. |
| src/drivers/tcp.c | Adds per-conn spinlock protection and connect timeout enforcement. |
| src/drivers/sound.c | Adds bounds check for malformed WAV chunk sizes. |
| src/drivers/sdcard.h | Adds ensure_ready, delete_recursive, and sd_set_slow_mode APIs. |
| src/drivers/sdcard.c | Adds rate-limited corruption logging, ensure_ready remount probe, fopen remount+retry, recursive delete. |
| src/drivers/mp3_player.c | Converts ring indices to atomics with explicit memory order. |
| src/drivers/http.h | Declares http_check_timeouts. |
| src/drivers/http.c | Adds recv draining on poll, recv overflow guard, timeouts, and switches Mongoose allocator hooks. |
| src/drivers/fileplayer.c | Exits early if WAV buffer allocation fails. |
| src/drivers/display.h | Declares new raycasting-oriented display primitives. |
| src/drivers/display.c | Implements fillVLine, drawTexturedColumn, fillVLineGradient. |
| simulator/unicorn_trampolines.c | Adds FS ops slots, nested-callback return mechanism, debug instrumentation, HTTP isComplete slot. |
| simulator/unicorn_runner.c | Enlarges emulated memory, adds shadow mapping, nested callback stop/restart logic, mem error hook. |
| simulator/stubs/driver_stubs.c | Adds recursive delete via nftw, stubs for ensure_ready/slow_mode, more display diagnostics, updates native audio callback atomic type. |
| simulator/sim_wifi.c | Updates simulator WiFi to report ONLINE; adds hasInternet and isHwDisconnected stub. |
| simulator/sim_socket_handler.c | Forces stb PNG path, adds screenshot debug, adds display diagnostics RPC handlers. |
| simulator/main.c | Initializes toast system before networking; updates HTTP setReadBufferSize signature. |
| simulator/CMakeLists.txt | Adds toast + zip bridge + miniz sources/includes and miniz compile definitions. |
| sdk/native/terminal.h | Expands terminal API (line numbers, scrollbar, wrap, bounds). |
| sdk/native/os.h | Mirrors OS API struct expansions (display/fs/http/wifi) and atomic audio callback. |
| CMakeLists.txt | Adds toast/core1_alloc/zip bridge + miniz sources; sets miniz compile flags and forced include config. |
| apps/wikipedia/app.json | Adds category/repo/min_firmware metadata. |
| apps/systest/app.json | Adds category/repo/min_firmware metadata. |
| apps/snake/app.json | Adds category/repo/min_firmware metadata. |
| apps/picoforge/main.lua | Switches terminal font selection to string name. |
| apps/picoforge/app.json | Adds category/repo/min_firmware metadata. |
| apps/hello/app.json | Adds category/repo/min_firmware metadata. |
| apps/hello_c/app.json | Reformats JSON and adds category/repo/min_firmware metadata. |
| apps/graphicstest/app.json | Adds category/repo/min_firmware metadata. |
| apps/gbc/main.c | Updates listDir callback signature; improves CGB palette handling + input loop placement. |
| apps/gbc/display.h | Adds cached CGB LUT. |
| apps/gbc/display.c | Uses cached LUT to avoid per-pixel conversion; adds LUT updater. |
| apps/gbc/app.json | Adds category/repo/min_firmware metadata. |
| apps/filemanager/app.json | Adds category/repo/min_firmware metadata. |
| apps/editor/app.json | Adds category/repo/min_firmware metadata. |
| apps/doom/dg_picos.c | Feeds watchdog via sys->poll during waits / init timing calls. |
| apps/doom/app.json | Adds category/repo/min_firmware metadata. |
| apps/burnin_fixer/app.json | Adds category/repo/min_firmware metadata. |
| apps/block.exe/app.json | Adds category/repo/min_firmware metadata. |
| apps/3dtest/main.lua | Updates demo to PicoCalc model + fill modes; uses new 3D bridge API. |
| apps/3dtest/app.json | Adds category/repo/min_firmware metadata. |
| .gitmodules | Adds miniz and cdogs submodules. |
| .gitignore | Adds ignores for binary assets, screenshots, saves, and additional app artifacts. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| extern volatile bool g_core1_pause; | ||
| extern volatile bool g_core1_paused; | ||
|
|
There was a problem hiding this comment.
g_core1_pause/g_core1_paused are declared here as volatile bool, but their definitions were changed to _Atomic bool (see src/main.c). These declarations are not type-compatible and can cause compile/link issues or undefined behavior. Update these externs to _Atomic bool and include <stdatomic.h> (guarded appropriately) so the types match across translation units.
| // Update status immediately so callers see DISCONNECTED right away | ||
| s_status = WIFI_STATUS_DISCONNECTED; | ||
| atomic_store_explicit(&s_hw_disconnected, false, memory_order_release); | ||
| s_ssid[0] = '\0'; | ||
| s_ip[0] = '\0'; | ||
| printf("WiFi: disconnect queued\n"); |
There was a problem hiding this comment.
wifi_disconnect() clears s_ssid/s_ip without taking s_state_lock, but other code now reads/writes those fields under that spinlock. This introduces a cross-core data race (torn reads / inconsistent strings). Wrap the s_ssid/s_ip updates in the same spin_lock_blocking(s_state_lock) used in wifi_connect() / wifi_get_ssid() / wifi_get_ip().
| // ── Custom Mongoose Allocator ──────────────────────────────────────────────── | ||
| void *mg_calloc(size_t count, size_t size) { return umm_calloc(count, size); } | ||
| void *mg_calloc(size_t count, size_t size) { return core1_calloc(count, size); } | ||
|
|
||
| void mg_free(void *ptr) { umm_free(ptr); } | ||
| void mg_free(void *ptr) { core1_free(ptr); } |
There was a problem hiding this comment.
mg_calloc/mg_free were switched to core1_calloc/core1_free, but core1_alloc_init() is only called when a Core 1 pool exists (RP2350). On simulator / non-RP2350 builds, the pool is never initialized, so Mongoose allocations will dereference an uninitialized pool pointer and crash. Add a fallback to umm_calloc/umm_free (or another allocator) when the Core 1 pool is not configured/initialized.
| // Call progress callback if provided | ||
| if (has_progress) { | ||
| lua_pushvalue(L, 3); // push the callback | ||
| lua_pushinteger(L, files_done); | ||
| lua_pushinteger(L, actual_files); | ||
| lua_pcall(L, 2, 0, 0); | ||
| } |
There was a problem hiding this comment.
The progress callback is invoked with lua_pcall but the return value is ignored. On error, Lua leaves an error object on the stack, which will accumulate across iterations and corrupt subsequent Lua execution. Check the lua_pcall result and either (a) pop the error and abort extraction with a returned error string, or (b) pop the error and continue, but ensure the stack is balanced.
| // Build full output path | ||
| size_t dest_len = strlen(dest_dir); | ||
| bool needs_slash = dest_dir[dest_len - 1] != '/'; | ||
| snprintf(full_path, sizeof(full_path), "%s%s%s", | ||
| dest_dir, needs_slash ? "/" : "", stat.m_filename); | ||
|
|
There was a problem hiding this comment.
snprintf(full_path, ...) truncation is not checked. If dest_dir + filename exceeds MAX_EXTRACT_PATH, extraction will silently write to a truncated path (wrong location / potential overwrite). Detect truncation (snprintf return value) and fail the extraction (or skip the entry with an error).
| bool wifi_hw_disconnected(void) { | ||
| return false; | ||
| } | ||
|
|
There was a problem hiding this comment.
wifi_hw_disconnected() always returns false in the simulator. Any Lua/app code that waits for picocalc.network.isHwDisconnected() before continuing (e.g. voltage-sensitive SD writes) will hang forever in the simulator. Return true when the simulated WiFi state is disconnected (or after a queued disconnect is processed).
| // SD card stubs | ||
| void sdcard_remount(void) {} | ||
| void sdcard_apply_clock(void) {} | ||
| bool sdcard_ensure_ready(void) { return true; } | ||
| void sd_set_slow_mode(bool slow) { (void)slow; } |
There was a problem hiding this comment.
Simulator stub defines sdcard_remount() as returning void, but the real API returns bool (see src/drivers/sdcard.h). Callers may read an undefined return value in simulator builds. Update the stub signature to return bool and return an appropriate success/failure value.
This pull request introduces significant enhancements across the platform, focusing on improved stability, expanded core features, and a more polished user experience. It also includes several internal refactorings and simulator improvements for better development and testing.
Key Improvements:
fs.ensureReady()for proactive SD card health checks, andfs.setSlowMode()for reliable writes during WiFi activity.fillVLine,drawTexturedColumn, andfillVLineGradientfor high-performance 3D rendering.sys.getMemInfo()to include detailed information on PIO PSRAM and XIP cache hit rates for better system monitoring.app.jsonschema. Improves app description scrolling and overall navigation.editorwith terminal SDK features (word wrap, line numbers, font cycling) and adds various stability and feature improvements tosystest,updater, andpicoforgeapplications.This comprehensive update aims to make the platform more stable, feature-rich, and user-friendly, while also providing a more robust environment for developers.