Enhance the Full Story bug reporting process

[VictoriaExpensify]

Background:
Full Story is a powerful tool that provides valuable insights into how customers interact with the product and the challenges they encounter. It allows us to identify bugs even when they aren't explicitly reported by customers, offering a critical opportunity to address and fix issues quickly.

Problem:
Currently, reporting bugs discovered through Full Story can be challenging. The Research Team must manually track down the issue within the logs, a task that is both technical and time-consuming. This process becomes even more challenging when the bug cannot be easily reproduced, leading to delays in addressing the problem.

Solution:
We need to streamline and simplify the bug-reporting process from Full Story. Potential solutions could include:

• Create a "how to" guide for locating the bug within our logs
• Use OpenAI to create a job that can isolate the bug within the logs
• Explore FS development tools to see what opportunities there are to make capturing bug logs easier (this information is not currently available in FS, but it may be possible to pull it on)
• When a bug report is created from FS, use a bot to cross-reference existing issues to see if this is already known
• Define a process for when to create a bug report vs when to create a tracking issue (i.e. if a bug cannot be reproduced, create a tracking issue that gets cross-referenced using the above process)

[VictoriaExpensify]

Here are some examples of bugs that have been viewed through FS, but could not be reproduced and the root cause could not be found:

• https://expensify.slack.com/archives/C07HPDRELLD/p1733784607625009
• Hmm..it's not here page displayed when add a payment card #53173
• https://expensify.slack.com/archives/C07HPDRELLD/p1733781028374659

[VictoriaExpensify]

@kadiealexander and I have been brainstorming and think a good, achievable place to start here is to:

• Create a simple guide on how to read logs and identify loglines where bugs are occurring
• Add some automation for bugs check if bugs have previously been reported and, if not, add some automation to report them. This is how we see that working to start with:

1. Full Story flags error (either by console error or custom event, or team flagging manually via note)
2. Reference existing GH issues to see if it already exists (Zapier)
3. If bug exists, add bug report to existing tracking issue
4. If this is the first instance post bug in #expensify-bugs for Applause testers

We think there are other opportunities with FS and ChatGPT that we can keep exploring to improve these processes, but those two items are a good way to start getting us in the right direction pretty quickly.

@danielrvidal and @garrettmknight - is this along the lines of what you had in mind and do you think this would be helpful?

[melvin-bot]

@garrettmknight, @danielrvidal, @VictoriaExpensify, @kadiealexander Uh oh! This issue is overdue by 2 days. Don't forget to update your issues!

[VictoriaExpensify]

Not OD

[garrettmknight]

I like the direction you're going. A few notes on the log search side of things from a recent chat with @trjExpensify:

Order of opps:

1. Narrow the time as much as you can
2. Add blobs to find typical errors. Hmmm, EXPError, Error, Invalid
3. Search exp org in GH for the logged error to find the command
4. If this finds the command, try to understand the command and input
5. Check the logs for the inputs to see what might have failed

If you can't find the command from the error:

• Try reproducing for fresh logs and follow the network tab to find out what we call
• Search around for the command that you think the user was calling

Other tips:

• Git blame to see if anything was touched recently
• Check the deploy checklist for App errors

[garrettmknight]

Reference existing GH issues to see if it already exists (Zapier)

I think this would be a really great improvement to our QA processes in general, not just for FS reporting.

[VictoriaExpensify]

I think this would be a really great improvement to our QA processes in general, not just for FS reporting.

100%!

[VictoriaExpensify]

Not OD but Kadie and I are both ooo for the holidays now. We will pick this up when we return

[melvin-bot]

@garrettmknight, @danielrvidal, @VictoriaExpensify, @kadiealexander Uh oh! This issue is overdue by 2 days. Don't forget to update your issues!

[danielrvidal]

I don't think this is urgent/overdue, so I'll move it to weekly until we're back.