feat: catch errors from web container and show in actionable alert so user can send them to AI for fixing

[wonderwhy-er]

Summary

This PR introduces enhancements to error handling for web container previews

Key Changes

Error Handling for Previews:

• Added error forwarding from web container previews (preview-message event listener).
• Differentiated handling for uncaught exceptions and unhandled promise rejections.
• Triggered actionable error alerts in the UI when preview errors occur.

User Interface Improvements:
Updated ChatAlert component to display context-specific error messages for terminal and preview errors.

Added changes to readme

[faddy19]

I made a small drawing.
I agree that it has to be integrated with the Terminal Error system but there are several steps we need to consider.

1. We need to capture all Errors appearing in Browser Console and feed them back to the LLM exactly the same way as for the Terminal.

2. We have different categories of Errors and they come early and delayed. So the when and where of each error is important.We need that.

3. We have to recheck if new errors came in and make a decision if we feed it into the LLM automatically or let the user decide by Clicking Fix Error.

I would start the with the most challenging: Capture all errors from all Containers and Browser Console and save it to Event Logs. Right now Event Logs is missing 70% of errors. I checked that.

[faddy19]

I was not able to capture these errors and feed it back to the LLM. As there are not only Terminal Errors but also errors only appearing when rendered in the browser. We might also include these into the fix errors component. Let me know what you think

[faddy19]

[faddy19]

I have have different approaches:

We need to identify which container each error originates from.

Unique Iframe IDs: Instead of relying solely on the source in the postMessage, which can be ambiguous, let's use unique IDs for each Webcontainer's iframe. This way, you can be sure of the origin of the log. This could be a simple sequential number or a more sophisticated identifier.
postMessage with Iframe ID: Include the iframe's ID in every postMessage sent from the Webcontainer.

Stringify Error Objects: Instead of just getting the error message, try to serialize the entire Error object, including the stack trace when available (for exceptions). This often provides much more context.
Capture Error.stack: Error.stack is a browser feature which often contains useful stack trace information.

Browser Extension Assistance:

Content Script: If you absolutely need a very high level of certainty, a browser extension's content script might be able to intercept console messages and send them to the background script which, in turn, can send it to a web page. This goes beyond the usual restrictions of a web page's script. But this means user need to install an extension
DevTools Protocol: A browser extension can use the DevTools protocol to access browser console logs, though this method is rather complex. This approach is not "from within" the application, and requires the user to install an extension.

Inline script in : Ensure that the script that overrides console.error, window.onerror, and sets up the postMessage listener within the iframe is in an inline script in the iframe's . This is the best way to ensure that it runs as early as possible to intercept errors that may occur during page load.
Avoid async or deferred scripts: Ensure your script in the iframe does not use async or defer attributes on the script tag, since they will load late.

Use Web Worker (for some web container implementations):

If your web container implementation uses a web worker instead of iframe, the errors in worker are not caught by window.onerror. You have to use worker.onerror to catch errors in the web worker.

Introduce a Centralized Logging Function (Within the Iframe):

Unified Function: In the iframe, instead of calling window.parent.postMessage directly, call a centralized logging function inside your iframe that calls postMessage, but also include the iframe id, so if postMessage is somehow blocked or lost, you can add a backup method in the log function
7. Server-Side Logging (As a Backup/Complement):

Fallback: Even with browser methods, sending a copy of the logs to a server-side endpoint offers a backup if there's a rare scenario where client-side capturing fails.
More sophisticated approach: Instead of calling your logging system in the main application, you may implement an endpoint, so the web container makes a direct call to your logging endpoint. This also guarantees log capture if postMessage failed.

[wonderwhy-er]

I have have different approaches:

We need to identify which container each error originates from.

Unique Iframe IDs: Instead of relying solely on the source in the postMessage, which can be ambiguous, let's use unique IDs for each Webcontainer's iframe. This way, you can be sure of the origin of the log. This could be a simple sequential number or a more sophisticated identifier. postMessage with Iframe ID: Include the iframe's ID in every postMessage sent from the Webcontainer.

Stringify Error Objects: Instead of just getting the error message, try to serialize the entire Error object, including the stack trace when available (for exceptions). This often provides much more context. Capture Error.stack: Error.stack is a browser feature which often contains useful stack trace information.

Browser Extension Assistance:

Content Script: If you absolutely need a very high level of certainty, a browser extension's content script might be able to intercept console messages and send them to the background script which, in turn, can send it to a web page. This goes beyond the usual restrictions of a web page's script. But this means user need to install an extension DevTools Protocol: A browser extension can use the DevTools protocol to access browser console logs, though this method is rather complex. This approach is not "from within" the application, and requires the user to install an extension.

Inline script in : Ensure that the script that overrides console.error, window.onerror, and sets up the postMessage listener within the iframe is in an inline script in the iframe's . This is the best way to ensure that it runs as early as possible to intercept errors that may occur during page load. Avoid async or deferred scripts: Ensure your script in the iframe does not use async or defer attributes on the script tag, since they will load late.

Use Web Worker (for some web container implementations):

If your web container implementation uses a web worker instead of iframe, the errors in worker are not caught by window.onerror. You have to use worker.onerror to catch errors in the web worker.

Introduce a Centralized Logging Function (Within the Iframe):

Unified Function: In the iframe, instead of calling window.parent.postMessage directly, call a centralized logging function inside your iframe that calls postMessage, but also include the iframe id, so if postMessage is somehow blocked or lost, you can add a backup method in the log function 7. Server-Side Logging (As a Backup/Complement):

Fallback: Even with browser methods, sending a copy of the logs to a server-side endpoint offers a backup if there's a rare scenario where client-side capturing fails. More sophisticated approach: Instead of calling your logging system in the main application, you may implement an endpoint, so the web container makes a direct call to your logging endpoint. This also guarantees log capture if postMessage failed.

I get strong vibes of ChatGPT writing this :D
Its written without knowing how it actually works.

1. There are no multiple iframes for ids.
   2.1 We do not control how WebContainers work, its closed source tech and we can't access and change what is happening in iframe
   2.2 there are only 2 ways to get errors from iframe. What I used here is what webcontainers already provide. Second approach is to instruct LLMs to make apps post their own error messages. That is finicky and unreliable.
   2.3 What this means is that from 3 possible approaches(switch from web containers, ask LLMs to post messages from iframe, or use messages already sent by webcontainers) using approach I did here is lowest hanging fruite giving us 80% of result for 20% of effort. We can revisit this if we ever switch from webcontainers to something else.

[thecodacus]

is this line showing all the console logs from preview??

webcontainer.on('preview-message', (message) => {
          console.log('WebContainer preview message:', message);
        });

[faddy19]

One is for sure. We do not need to dive into the Webcontainer even it is closed source. At some point we should get rid of closed source as we cannot control the inside.

We see these errors in browsers console and can grab them. What we can see we should be able to programmatically get.

My ideas here are:

1. Create a small Browser Extension which reads these logs in near realtime and feeds it into our LLm to fix the issue.
   The Browser Extension with Real-Time Log Streaming & LLM Integration works like this:

Core Function: The extension acts as a real-time console log streamer. It intercepts console.log, console.warn, console.error, and other relevant output, as it appears in the browser's DevTools console.

Leveraging the DevTools Protocol from a Browser Extension:
DevTools Access: A browser extension has permission to use the DevTools Protocol (CDP), allowing you to directly access the same information that you see in the DevTools panel.
CDP Console API: The CDP exposes a Console domain that can provide log entries. The extension can use this API to receive console logs in a structured format, including the log level, message, and location information.

3. Proxying the Webcontainer API Endpoint (If Possible):

Intercept Communication: Instead of trying to capture the logs after they are generated, we might be able to capture them before they even reach the browser's console by intercepting the communication

What do you guys think?

[faddy19]

is this line showing all the console logs from preview??

webcontainer.on('preview-message', (message) => {

          console.log('WebContainer preview message:', message);

        });

I tried and did not get all errors with that.

[thecodacus]

I tried and did not get all errors with that.

not all errors are from preview, some errors are from bolt itself which we can ignore as those are not related to the project that is being built.

we just need the errors that are originated from the preview window only

[faddy19]

@thecodacus absolutely right.
Catching all errors is a challenge. I tried 10 different approaches locally but failed.

Anyone more capable of capturing them. Will try tomorrow and update you guys

Maybe we can join forces as this preview issue seems to be a big road blocker

[wonderwhy-er]

is this line showing all the console logs from preview??

webcontainer.on('preview-message', (message) => {

          console.log('WebContainer preview message:', message);

        });

I tried and did not get all errors with that.

What errors did you try?
Here i gist of chat.
It does catch javascript errors
And also unhandled promises
Does not capture scripts that failed to load
That we will need to handle on prompt side to capture and rethrow as error that webcontainer can capture and send our way.

What you proposed before with extensions is not feasible.

Can you share gist of exported chat that shows what kinds of errors it does not catch?

[wonderwhy-er]

Intercept Communication: Instead of trying to capture the logs after they are generated, we might be able to capture them before they even reach the browser's console by intercepting the communication

That will not really work because errors do not come from webcontainer.

What is happening is this:

1. Bolt.diy download webassembly webcontainers
2. Sends in files and runs terminal commands in it
3. Webcontainer connects to stackblitz services to as gateway to provide url we can then open in iframe
4. IFrame loads webpage from stackblitz that comes from webassembly in our browser
5. Errors happen in it and are captured by some code that stackblitz injects, and then posts message
6. We capture it

That iframe is on other domain we do not control, so we can't really go in and capture things ourselves
We do have control over what page is there, so we can try injecting something to get us more control over that. But that is tricky to do... As projects can use variety of server frameworks to expose webpages and we do not know how to inject our error handling code into it

So we get back to what I shared before.
We have 3 options:

1. Change prompts to force AI to generate services with injected error handling code(unreliable as LLM may fail to do that)
2. Change webcotnaienr tech to one we can control.(Really big effort)
3. Use what web containers already provide which I did her.

[wonderwhy-er]

Ouh wait, just found something ineresting
https://webcontainers.io/api#%E2%96%B8-setpreviewscript

We can ask webcontainer to inject script that will be loaded in all preview urls that are open.
So if you can share what errors you want to catch we can write script that catches them and rethrows them in a way we can then catch on our side

[thecodacus]

then we can also monkeypatch the console.log to have additional function calls, and replace the original one with our modified one
here is what claude shows how to do it
https://claude.site/artifacts/2f5dee75-00e2-4426-939d-647e67022a73

and this is what it says about intercepting the errors

// Store the original error handling function
const originalOnError = window.onerror;
const originalUnhandledRejection = window.onunhandledrejection;

// Global error handler
window.onerror = function(message, source, lineno, colno, error) {
    const errorInfo = {
        message,
        source,
        lineno,
        colno,
        timestamp: new Date().toISOString(),
        type: 'uncaught-error'
    };

    try {
        // Process the error
        processError(error, errorInfo);
    } catch (processingError) {
        console.error('Error in error processing:', processingError);
    }

    // Call the original handler if it exists
    if (originalOnError) {
        return originalOnError.apply(this, arguments);
    }

    // Rethrow the error by default
    return false;
};

// Handle Promise rejections
window.onunhandledrejection = function(event) {
    const errorInfo = {
        message: event.reason?.message || 'Unhandled Promise Rejection',
        timestamp: new Date().toISOString(),
        type: 'unhandled-rejection'
    };

    try {
        // Process the error
        processError(event.reason, errorInfo);
    } catch (processingError) {
        console.error('Error in promise rejection processing:', processingError);
    }

    // Call the original handler if it exists
    if (originalUnhandledRejection) {
        return originalUnhandledRejection.apply(this, arguments);
    }

    // Prevent the default handling
    event.preventDefault();
};

[faddy19]

Guys that is really high level. Thanks for the input.

I must say that this closed source webcontainer thing is really something no developer likes but tolerates. At some point we can replace that but not now.

[wonderwhy-er]

then we can also monkeypatch the console.log to have additional function calls, and replace the original one with our modified one here is what claude shows how to do it https://claude.site/artifacts/2f5dee75-00e2-4426-939d-647e67022a73

and this is what it says about intercepting the errors

// Store the original error handling function
const originalOnError = window.onerror;
const originalUnhandledRejection = window.onunhandledrejection;

// Global error handler
window.onerror = function(message, source, lineno, colno, error) {
    const errorInfo = {
        message,
        source,
        lineno,
        colno,
        timestamp: new Date().toISOString(),
        type: 'uncaught-error'
    };

    try {
        // Process the error
        processError(error, errorInfo);
    } catch (processingError) {
        console.error('Error in error processing:', processingError);
    }

    // Call the original handler if it exists
    if (originalOnError) {
        return originalOnError.apply(this, arguments);
    }

    // Rethrow the error by default
    return false;
};

// Handle Promise rejections
window.onunhandledrejection = function(event) {
    const errorInfo = {
        message: event.reason?.message || 'Unhandled Promise Rejection',
        timestamp: new Date().toISOString(),
        type: 'unhandled-rejection'
    };

    try {
        // Process the error
        processError(event.reason, errorInfo);
    } catch (processingError) {
        console.error('Error in promise rejection processing:', processingError);
    }

    // Call the original handler if it exists
    if (originalUnhandledRejection) {
        return originalUnhandledRejection.apply(this, arguments);
    }

    // Prevent the default handling
    event.preventDefault();
};

This is already returned, I already showed in this PR that webcontainer returns unhandled rejections and errors.
That is why I asked what else is missing. Because if we care only about these two then it works.

[thecodacus]

This is already returned, I already showed in this PR that webcontainer returns unhandled rejections and errors. That is why I asked what else is missing. Because if we care only about these two then it works.

I guess we will start with this. let see how it looks in the UI and the full flow, then we will be able to see if its missing any thing

[faddy19]

I posted this earlier. There are react errors in the app inside webcontainer. Also other errors.
@wonderwhy-er @thecodacus Will these be captured as well?

The best would be to combine the existing fix error logic in the Terminal pull request with this one so we do not need to implement a new fixing logic. Inside we can ask LLm to fix both as these are running in different containers. We could do 2 in one.

Is there a way we could talk also not just write? That would be awesome

[faddy19]

I am going to list the categories of errors and we can start anticipating which can appear. I see bolt fails

List of possible errors:

Error List:

• Webcontainer Failure
• Iframe Loading Errors
• Preview Errors
• Webcontainer Core Resource Loading Failures
• Contextify Errors
• Terminal Launch Failures
• Shell Command Errors
• Shell Command Timeouts
• Shell Command Output Errors
• Missing dependencies errors
• File Creation Errors
• File Writing Errors
• File Reading Errors
• File System Operation Timeouts
• JavaScript Runtime Errors
• JavaScript Import/Module Load Errors
• Unhandled Promise Rejections
  console.log, console.warn, console.error, console.info logs
• Web Worker Errors
• React Component Errors
• "React is not defined" Errors
  postMessage Failures
• Cross-Context Message Handling Issues
• Webcontainer API Communication Failures
• Webcontainer Network Call Failures
• Chat Console API Errors
• LLM API Errors
• File System Permission Errors
• File System Quota Errors
• File Not Found Errors
• File Corruption Errors
• Network Errors During File Transfers
• Encoding Errors
• Authentication and Authorization Failures
• Github Import/Sync Errors

[faddy19]

Error handling ist the most critical thing here. I tested bolt.new extensively. It fails too many times because error handling is not working very well.

Step 1: Error Capturing
Step 2: Error Handling and Fixing
Step 3: Self Healing
Step 4: Error Anticipating and Error Prevention before the Error is created 😀

[wonderwhy-er]

Hm, need to reproduce that react error and see if code I did here reacts to it.
A for most of other things you wrote.
I'd rather spent time fixing errors that happen than spend time fixing errors that did not happen.

So, want something captured? Share gist of a chat where failure happens but is not detected.
Otherwise I have nothing to look at beside hypotheticals.

[wonderwhy-er]

Connected with merged @thecodacus terminal errors code
Can you guys give it a try
Does this catch errors you had in mind @faddy19 ?

[thecodacus]

export type ActionType = 'file' | 'shell' | 'preview'; is this change required? i don't see it being used anywhere and relevant

[thecodacus]

Connected with merged @thecodacus terminal errors code Can you guys give it a try Does this catch errors you had in mind @faddy19 ?

I am gonna test this now. will update once tested with different errors

[faddy19]

Connected with merged @thecodacus terminal errors code Can you guys give it a try Does this catch errors you had in mind @faddy19 ?

I still have the React Error which is not captured by the Error handling system

Here is a screenshot

Here is the Chat export

all-chats-2024-12-24T08_13_11.308Z.json

[faddy19]

Terminal errors work very good, great job guys.

One thing for the terminal errors feedback. If you cancel the current command in terminal, the Terminal error is showing and you can let it fix, but have to start the npm run dev on your own. There should be an addition to run npm run dev in the end.

[faddy19]

Connected with merged @thecodacus terminal errors code Can you guys give it a try Does this catch errors you had in mind @faddy19 ?

I still have the React Error which is not captured by the Error handling system

Here is a screenshot

Here is the Chat export

all-chats-2024-12-24T08_13_11.308Z.json

That was the part I meant earlier. If we can get the errors we see in the console we can just send it to the LLM and let it fix. What happens is there is an import React missing somewhere and it cannot run the Preview.

[thecodacus]

Terminal errors work very good, great job guys.

One thing for the terminal errors feedback. If you cancel the current command in terminal, the Terminal error is showing and you can let it fix, but have to start the npm run dev on your own. There should be an addition to run npm run dev in the end.

yes manually interrupting terminal command recognize it as error cuz it throws an error exit code. will try to find the exact error code for that and exclude it from throwing errors later

@wonderwhy-er the app is throwing error in my case

I believe below is the reason

reason is, this file is also imported in server code and I have seen app throwing error anytime where in server code an import is done where we have a nanostore created. even if we don't use it in the code in runtime just by importing it throws error

I suggest this change

[wonderwhy-er]

@thecodacus I just pushed two fixes you suggested.
You are right, that action change was not needed.

[wonderwhy-er]

@faddy19 I just tested on your chat and it worked in this PR
Undefined react errors is caught:

[faddy19]

Let me retest it. I might missed something. Thanks a lot for pointing this out

[thecodacus]

awesome great, one more change I think would be great, but maybe optional..
i wanted to stacktrace to point the actual files instead of webcontainer urls

here is what claude suggested

function cleanStackTrace(stackTrace: string): string {
  // Function to clean a single URL
  const cleanUrl = (url: string): string => {
    const regex = /^https?:\/\/[^\/]+\.webcontainer-api\.io(\/.*)?$/;
    if (!regex.test(url)) {
      return url;
    }
    const pathRegex = /^https?:\/\/[^\/]+\.webcontainer-api\.io\/(.*?)$/;
    const match = url.match(pathRegex);
    return match?.[1] || '';
  };

  // Split the stack trace into lines and process each line
  return stackTrace.split('\n').map(line => {
    // Match any URL in the line that contains webcontainer-api.io
    return line.replace(
      /(https?:\/\/[^\/]+\.webcontainer-api\.io\/[^\s\)]+)/g,
      match => cleanUrl(match)
    );
  }).join('\n');
}

Test it with your stack trace

const stackTrace = `Error
    at printWarning (https://6toq0kapqpmjiyrdqgbkv347gyeld0-2tu1--5173--c8c182a3.local-corp.webcontainer-api.io/node_modules/.vite/deps/react-dom.js?v=9d11fbfb:519:38)
    at error (https://6toq0kapqpmjiyrdqgbkv347gyeld0-2tu1--5173--c8c182a3.local-corp.webcontainer-api.io/node_modules/.vite/deps/react-dom.js?v=9d11fbfb:503:15)
    at Object.render (https://6toq0kapqpmjiyrdqgbkv347gyeld0-2tu1--5173--c8c182a3.local-corp.webcontainer-api.io/node_modules/.vite/deps/react-dom.js?v=9d11fbfb:21444:13)
    at https://6toq0kapqpmjiyrdqgbkv347gyeld0-2tu1--5173--c8c182a3.local-corp.webcontainer-api.io/src/main.jsx:4:10`;

console.log(cleanStackTrace(stackTrace));

This will output:

Error
    at printWarning (node_modules/.vite/deps/react-dom.js?v=9d11fbfb:519:38)
    at error (node_modules/.vite/deps/react-dom.js?v=9d11fbfb:503:15)
    at Object.render (node_modules/.vite/deps/react-dom.js?v=9d11fbfb:21444:13)
    at src/main.jsx:4:10

[wonderwhy-er]

Good suggestions, pushed in url cleanup, tested on couple of examples quick, need to go now.
May need more testing.

[thecodacus]

yeah.. I will do some testing. and will merge if all look good

[faddy19]

Battle Tested the Error capturing guys. Works really good now. I have one more feedback
Sometimes it asks for options or y/n in terminal. Lets auto approve this as the user just want to contiunue.

[faddy19]

Next step would be that we automatically fix these errors in the background that we do not bother with Ask Bolt or Fix this. We want to have a preview. Or we implement a toggle in Settings and say autoapprove error fixing and Terminal commands. That would give control

What do you think?

[faddy19]

I think you can already push to main @thecodacus

[faddy19]

One moment . With this error Error Fixing did not appear.

[faddy19]

Also when page is relaoded and error is reloaded the Ask bolt error fixing does not appear

[faddy19]

One more idea: Would it make sense to put a panic button in there so if the preview or app get stuck, the user clicks it and we send a prompt to the llm to check for errors in terminal and the application and to fix it?

It is impossible to capture all errors and some standard user might not be able to copy paste the error

[wonderwhy-er]

One moment . With this error Error Fixing did not appear.

Can you share the just to take s look?

We can add experiment switch in separate PR to experiment with auto fixing
Windsurf can do it.

We can't really look into preview yet. May be We can with that script injection.
But I want to add a form of test driven workflow to bolt so we can do something like you described.

We will get there, step by step

[thecodacus]

Also when page is relaoded and error is reloaded the Ask bolt error fixing does not appear

this comes into the terminal, but since its not stopping the command so the error check does not trigger.
to identify this we have to separately monitor the running terminal outputs, for any error codes

[thecodacus]

for now I thing this PR is capturing the Preview error just fine. its good to merge