fix: Huge memory consumption

[EArminjon]

Is there an existing issue for this?

• I have searched the existing issues.

Version

0.4.1

Description

Our app have many, many golden test like following. The issue is about memory consumption, we are today at 40go every run. Do you know what in golden test took so much memory ? Any general good practice ?

 goldenTest(
      'renders texts correctly',
      fileName:
          'theme-texts-${flavor.runtimeType.toString().toLowerCase()}-${mode.name}',
      builder: () => GoldenTestGroup(
        columns: 1,
        children: <GoldenTestScenario>[
          GoldenTestScenario(
            name: 'body text 1',
            child: Text(
              'Sample text',
              style: textTheme.bodyText1,
            ),
          ),
          GoldenTestScenario(
            name: 'body text 2',
            child: Text(
              'Sample text',
              style: textTheme.bodyText2,
            ),
          ),
          GoldenTestScenario(
            name: 'headline 1',
            child: Text(
              'Sample text',
              style: textTheme.headline1,
            ),
          ),
          GoldenTestScenario(
            name: 'headline 2',
            child: Text(
              'Sample text',
              style: textTheme.headline2,
            ),
          ),
          GoldenTestScenario(
            name: 'headline 3',
            child: Text(
              'Sample text',
              style: textTheme.headline3,
            ),
          ),
          GoldenTestScenario(
            name: 'headline 4',
            child: Text(
              'Sample text',
              style: textTheme.headline4,
            ),
          ),
          GoldenTestScenario(
            name: 'headline 5',
            child: Text(
              'Sample text',
              style: textTheme.headline5,
            ),
          ),
          GoldenTestScenario(
            name: 'headline 6',
            child: Text(
              'Sample text',
              style: textTheme.headline6,
            ),
          ),
          GoldenTestScenario(
            name: 'subtitle 1',
            child: Text(
              'Sample text',
              style: textTheme.subtitle1,
            ),
          ),
          GoldenTestScenario(
            name: 'subtitle 2',
            child: Text(
              'Sample text',
              style: textTheme.subtitle2,
            ),
          ),
        ],
      ),
    );

Steps to reproduce

run many golden test

Expected behavior

As less memory as possible

Screenshots

No response

Additional context and comments

No response

[Kirpal]

Does this happen when you run the provided example or when you run all the tests in your app at once? I'm unable to reproduce using the example.

[EArminjon]

Does this happen when you run the provided example or when you run all the tests in your app at once? I'm unable to reproduce using the example.

Will try tomorrow. We have 104 golden test and would increase a lot the number...

[EArminjon]

Example consume few giga when copy paste >100 test but not 40g. I need to find where in our app we have the issue. @Kirpal , Is their any practice to absolutely avoid when doing golden test ? Must i cache something ?

[EArminjon]

@Kirpal this example use 10go (in our app we used 3 other similar process which increase the total near 40go)
The goal of this test which i simplified is to run many golden test for Light and Dark theme for each Flavor of our app. I remove here the flavor logic and just use a simple List.generate to mimic.

import 'package:alchemist/alchemist.dart';
import 'package:flutter/material.dart';
import 'package:flutter_test/flutter_test.dart';
import 'package:golden_toolkit/golden_toolkit.dart';

final List<ThemeMode> modes = <ThemeMode>[ThemeMode.light, ThemeMode.dark];

void goldenForAllThemesAndModes(
  void Function(
    dynamic flavorConfig,
    ThemeMode themeMode,
    ThemeData themeData,
  )
      run,
) {
  for (final int index in List<int>.generate(13, (int index) => index)) {
    for (final ThemeMode mode in modes) {
      final ThemeData themeData =
          mode == ThemeMode.light ? ThemeData.light() : ThemeData.dark();
      AlchemistConfig.runWithConfig(
        config: AlchemistConfig(theme: themeData),
        run: () => run(index, mode, themeData),
      );
    }
  }
}

void main() {
  setUpAll(() async {
    await loadAppFonts();
  });

  void _textsTest(
    dynamic abstractConfig,
    ThemeMode mode,
    TextTheme textTheme,
  ) {
    goldenTest(
      'renders texts correctly',
      fileName:
          'theme-texts-${abstractConfig.toString().toLowerCase()}-${mode.name}',
      builder: () => GoldenTestGroup(
        columns: 1,
        children: <GoldenTestScenario>[
          GoldenTestScenario(
            name: 'body text 1',
            child: Text(
              'Sample text',
              style: textTheme.bodyText1,
            ),
          ),
          GoldenTestScenario(
            name: 'body text 2',
            child: Text(
              'Sample text',
              style: textTheme.bodyText2,
            ),
          ),
          GoldenTestScenario(
            name: 'headline 1',
            child: Text(
              'Sample text',
              style: textTheme.headline1,
            ),
          ),
          GoldenTestScenario(
            name: 'headline 2',
            child: Text(
              'Sample text',
              style: textTheme.headline2,
            ),
          ),
          GoldenTestScenario(
            name: 'headline 3',
            child: Text(
              'Sample text',
              style: textTheme.headline3,
            ),
          ),
          GoldenTestScenario(
            name: 'headline 4',
            child: Text(
              'Sample text',
              style: textTheme.headline4,
            ),
          ),
          GoldenTestScenario(
            name: 'headline 5',
            child: Text(
              'Sample text',
              style: textTheme.headline5,
            ),
          ),
          GoldenTestScenario(
            name: 'headline 6',
            child: Text(
              'Sample text',
              style: textTheme.headline6,
            ),
          ),
          GoldenTestScenario(
            name: 'subtitle 1',
            child: Text(
              'Sample text',
              style: textTheme.subtitle1,
            ),
          ),
          GoldenTestScenario(
            name: 'subtitle 2',
            child: Text(
              'Sample text',
              style: textTheme.subtitle2,
            ),
          ),
        ],
      ),
    );
  }

  group('theme', () {
    goldenForAllThemesAndModes(
      (
        dynamic abstractConfig,
        ThemeMode themeMode,
        ThemeData themeData,
      ) {
        _textsTest(abstractConfig, themeMode, themeData.textTheme);
      },
    );
  });
}

[EArminjon]

hi @Kirpal,

Do you well achieve to reproduce the issue ?

[EArminjon]

@Kirpal have you got time to check :'( ?

[EArminjon]

Hi @Kirpal ,

Any update :'( ?

[EArminjon]

up

[FufferKS]

I think I'm running into the same issue. My reproducible example below. Raw flutter test does not break a sweat. Alchemist version makes my macbook freeze after a few runs.

`enum HomeGoldenType { raw, alchemist }`

  for (var i = 0; i < 1000; i++) {
    await runTheGolden(i, HomeGoldenType.raw);
  }
  
  Future<void> runTheGolden(int iter, HomeGoldenType type) async {
    final testName = 'Home Iteration $iter';
    final fileName = 'home_iteration_$iter';
    final materialApp = MaterialApp(
      home: Builder(
        builder: (context) => ColoredBox(
          color: type == HomeGoldenType.raw ? Colors.red : Colors.blue,
          child: const Placeholder(),
        ),
      ),
    );
    if (type == HomeGoldenType.raw) {
      testWidgets(
        testName,
        (WidgetTester tester) async {
          debugPrint('running iteration $iter');
          await tester.pumpWidget(materialApp);
          await tester.pumpAndSettle();
          expect(find.byType(MaterialApp), matchesGoldenFile('$fileName.png'));
          debugPrint('finished iteration $iter');
        },
        tags: ['golden-expensiveness'],
      );
    } else if (type == HomeGoldenType.alchemist) {
      await goldenTest(
        testName,
        fileName: fileName,
        pumpBeforeTest: (t) {
          debugPrint('running iteration $iter');
          return t.pump();
        },
        whilePerforming: (tester) async {
          return () async {
            debugPrint('finished iteration $iter');
          };
        },
        constraints: BoxConstraints.tight(kDefaultTestViewport),
        builder: () => materialApp,
        tags: ['golden-expensiveness'],
      );
    }
  }

[EArminjon]

Check on your MacOS activity monitor app your RAM usage, if RAM is high, yep same issue :)

[FufferKS]

Oh yes it is