Autocomplete not always working

[Light2Dark]

Describe the bug

Kinda 2 issues I've seen, I think there are some improvements we can make to have the autocomplete powers of a Jupyter notebook.
There are some other issues but I saw on Discord there's some work on a native vscode plugin so I wonder what's the best way forward. Thanks!

1. Doesn't work all the time
   
   This is likely due to frontend/src/core/codemirror/completion/Autocompleter.ts

  asCompletionResult(
    position: number,
    message: CompletionResultMessage,
  ): CompletionResult {
    console.log("message", message)
    return {
      from: position - message.prefix_length,
      options: message.options.map((option) => {
        return {
          label: option.name,
          type: option.type,
          info: () => constructCompletionInfoNode(option.completion_info),
        };
      }),
      // validFor: /^\w*$/,
    };
  },

commenting out the validFor will give proper autocomplete.

2. Not showing docstrings upon '.'
   
   This is due to docstrings_limit at marimo/_runtime/complete.py#L316

def complete(
    request: CodeCompletionRequest,
    graph: dataflow.DirectedGraph,
    glbls: dict[str, Any],
    glbls_lock: threading.RLock,
    stream: Stream,
    docstrings_limit: int = 80, # Here, configuring it to something higher will help
    timeout: float | None = None,
    prefer_interpreter_completion: bool = False,
) -> None:

Similar issue: #2080

Environment

{
  "marimo": "0.9.20",
  "OS": "Darwin",
  "OS Version": "23.6.0",
  "Processor": "arm",
  "Python Version": "3.12.7",
  "Binaries": {
    "Browser": "131.0.6778.69",
    "Node": "v20.11.0"
  },
  "Dependencies": {
    "click": "8.1.3",
    "docutils": "0.21.2",
    "itsdangerous": "2.2.0",
    "jedi": "0.19.1",
    "markdown": "3.7",
    "narwhals": "1.12.1",
    "packaging": "24.1",
    "psutil": "6.1.0",
    "pygments": "2.18.0",
    "pymdown-extensions": "10.12",
    "pyyaml": "6.0.2",
    "ruff": "0.6.9",
    "starlette": "0.41.2",
    "tomlkit": "0.13.2",
    "typing-extensions": "4.12.2",
    "uvicorn": "0.32.0",
    "websockets": "12.0"
  },
  "Optional Dependencies": {
    "duckdb": "1.1.2",
    "pandas": "2.2.3",
    "polars": "1.12.0",
    "pyarrow": "18.0.0"
  }
}

Code to reproduce

No response

[mscolnick]

@Light2Dark im down for both of these changes. did you notice any issues when removing validFor: /^\w*$/,? i am find to remove that.

Not showing docstrings upon '.'

What exactly are you experiencing here? I sometimes do get completions on . but do see it not always showing up.

[Light2Dark]

did you notice any issues when removing validFor: /^\w*$/

yes, it's a little hard to debug, but I am finding some (like plotly commands) not autocompleting. Let me investigate, I'm not sure if it's because of this / a different bug.

What exactly are you experiencing here

I am expecting the docstring to show up as soon as I enter . like this:

I see the backend is returning empty completion docstrings when the number of options are too large, due to the docstrings_limit. This is likely why some libs like pandas / plotly have poor completions because the number of options can be very large.