Skip to content

Latest commit

 

History

History
121 lines (90 loc) · 4.24 KB

testing.md

File metadata and controls

121 lines (90 loc) · 4.24 KB
Raw

Plugin Smoke Testing

Manual tests to execute before plugin releases.

Setup

Pre-reqs: Run through the flutter setup and flutter getting started guides.

  • Run flutter upgrade in a terminal to get the latest version prior to starting testing.

Testing matrix

Testers should generally test against IntelliJ CE. We also support WebStorm and internal IntelliJ; we should have testers assigned to each for each roll.

Project Creation

Validate basic project creation.

  • Create a simple project (File > New > Project..., pick Flutter).
  • Confirm that:
    • Project contents are created.
      • Verify that a run configuration (main.dart) is enabled in the run/debug selector.
    • Navigation works.
      • Open lib/main.dart and navigate to ThemeData.
    • There are no analysis errors or warnings.
    • Pub operations work.
      • Open pubspec.yaml and click the "Packages get" and "Packages upgrade" links.
    • Flutter operations work.
      • From the Tools menu, select Flutter > Flutter Doctor.
    • Code completion works.
      • Change primarySwatch: Colors to some other color and validate that you get completions.

Project Import

Validate that an externally created project can be imported.

  • choose File > Open...
  • browse to and select <flutter-root>/examples/flutter_gallery
  • ensure there are no analysis errors or warnings
  • ensure that the flutter_gallery launch configuration shows up and is selected

Device Detection

Validate device selection.

  • (OS X) Verify that the simulator can be opened.
    • Disconnect all devices and quit the iOS simulator.
    • Ensure that a menu item to open the iOS simulator is enabled in the device pull down menu.
    • Select "Open iOS Simulator".
    • Verify that the simulator opens.

Run / Debug

Validate basic application running and debugging.

In the newly created app:

  • plug in an Android device (or open the iOS Simulator)
  • set a breakpoint on the _counter++ line
  • hit the 'debug' icon to start the app running
  • verify the app appears on the device
  • tap the '+' icon on the app
  • verify that the IDE pauses at the breakpoint, and that the Variables pane has the right value for _counter
  • hit resume in the debugger

Hot Reload

Validate basic hot reload functionality.

Assuming the app state from above (i.e., leave the Debug session running):

  • change the 'Button tapped ... times.' line to end in an exclamation point
  • change the _counter++ line to _counter--
  • hit the hot reload button in the debugger UI
  • validate that
    1. the state persisted (the same number of clicks in the UI), and
    2. the text changed to end in an exclamation point
    3. the + button decreases the value
  • change the text and counter line back
  • hit the Full Application Restart button
  • validate that the text and state resets, and count increases

Keybindings:

  • verify that the hot reload keybinding works (on a mac: cmd-option-; or cmd-\)

Debugging Sessions

Validate that a sequence of sessions works as expected.

After testing the above, terminate your debugging session and start another.

  • validate that a breakpoint is hit
  • verify that the reload keybinding works as expected

Project Open Verification

Verify that projects without Flutter project metadata open properly and are given the Flutter module type.

  • create a new Flutter project and delete IntelliJ metadata:
    • flutter create foo_bar
    • cd foo_bar
    • rm -rf .idea
    • rm foo_bar.iml
  • open project ("File > Open")
  • verify that the project has the Flutter module type (the device pull-down displays) and analyzes cleanly

Fresh Install Configuration

Verify installation and configuration in a fresh IDEA installation.

  • Follow the instructions to simulate a fresh installation.
  • (If not running in a "runtime workbench", install the plugins.)
  • Open "Languages & Frameworks>Flutter" in Preferences and verify that there is no Flutter SDK set.
  • Set the Flutter SDK path to a valid SDK location.
    • Verify that invalid locations are rejected (and cannot be applied).
  • Verify project creation, run/debug.