Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add native support for updating .terraform.lock.hcl #90

Merged
merged 32 commits into from
Jul 4, 2023
Merged

Commits on Jun 6, 2023

  1. Configuration menu
    Copy the full SHA
    9a44923 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    d184fe3 View commit details
    Browse the repository at this point in the history

Commits on Jun 7, 2023

  1. Configuration menu
    Copy the full SHA
    b38c67b View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    06704f5 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    222cba4 View commit details
    Browse the repository at this point in the history
  4. Validate SHASumsData

    minamijoyo committed Jun 7, 2023
    Configuration menu
    Copy the full SHA
    dd3df2b View commit details
    Browse the repository at this point in the history

Commits on Jun 15, 2023

  1. Implement an index of provider versions for caching

    Currently the Terraform Registry returns the zh hash for all platforms,
    but not the h1 hash, so the h1 hash has to be calculated separately.
    We need to calculate the values for each platform and merge the results.
    To avoid multiple downloads and recalculations for each directory, the
    results are cached in memory.
    minamijoyo committed Jun 15, 2023
    Configuration menu
    Copy the full SHA
    48abf5c View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    912c079 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    0585339 View commit details
    Browse the repository at this point in the history

Commits on Jun 16, 2023

  1. Configuration menu
    Copy the full SHA
    350423f View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    80face0 View commit details
    Browse the repository at this point in the history

Commits on Jun 21, 2023

  1. Implement version constraint detection

    In order to update dependency lock files, we need to detect the version
    constraints of the providers of the module, but the current Updater
    interface can only handle a single file. Therefore, we introduce a new
    concept called ModuleContext, which is supposed to provide the Updater
    with information of the module that is currently being updated. Changes
    to the Updater interface will be addressed later because it requires
    massive rewriting.
    
    Note that it does not actually re-implement the resolution of version
    constraints in terraform init. It is very simplified for the use we
    need. Version constraints only support simple constants and not
    comparison operators. Ignore what cannot be interpreted.
    minamijoyo committed Jun 21, 2023
    Configuration menu
    Copy the full SHA
    3d07683 View commit details
    Browse the repository at this point in the history
  2. Fix lint issues

    minamijoyo committed Jun 21, 2023
    Configuration menu
    Copy the full SHA
    721888c View commit details
    Browse the repository at this point in the history

Commits on Jun 22, 2023

  1. Inject a ModuleContext to Updater

    Until now, the Updater interface has been implemented statelessly, but
    in order to adding a feature for updating the dependency lock file, we
    also need the module context. I've only adjusted the interface for now,
    but this will also help us to achieve more smart rewriting in the
    future. We also need to access the index to cache the hash value of the
    provider, which is a global state between modules. I decided to access
    the global context via the module context because it is always needed in
    combination with the module context. In addition, the option and updater
    instances were also moved to the global context, as they have not
    changed within the process.
    minamijoyo committed Jun 22, 2023
    Configuration menu
    Copy the full SHA
    679f233 View commit details
    Browse the repository at this point in the history

Commits on Jun 26, 2023

  1. Add native support for updating .terraform.lock.hcl

    The Terraform dependency lock file (a.k.a. .terraform.lock.hcl) was
    introduced in v0.14. While this is fine for simple use cases, there are
    may challenges at scale. This is because terraform is based on
    per-directory processing, and updating lock files using the official
    terraform providers mirror and terraform providers lock commands is very
    inefficient.
    
    However, at that time, we avoid supporting for updating the lock file
    for the following reasons:
    
    - That the underlying reason for the complexity of updating the lock
      file is due to the lack of capability of the Terraform Registry; it
      does not return the h1 hash value.
    - That the Registry protocol could be improved in the future.
    - The lock file format implies to support not only providers but also
      modules.
    - The lock file format is an implementation detail.
    - It was easy to imagine that supporting multiple Terraform versions
      would be hard.
    
    It got worse as time went on:
    
    - With the rise of arm chips, the number of platforms required for
      locking has increased.
    - As our infrastructure grown, the number of directories has increased
      accordingly.
    
    However, no progress has been made on this issue in the upstream. So it
    is time to reinvent the wheel by ourselves. Calculate hash values in
    .terraform.lock.hcl and cache them to be able to update multiple
    directories at once.
    minamijoyo committed Jun 26, 2023
    Configuration menu
    Copy the full SHA
    ae1f023 View commit details
    Browse the repository at this point in the history
  2. Use filename instead of platform for keys of hash

    Initially, we thought of using the key of the zh hash as the platform,
    but we found out that it also includes metadata such as manifest.json,
    so we decided to use the filename as it is.
    minamijoyo committed Jun 26, 2023
    Configuration menu
    Copy the full SHA
    3dbe933 View commit details
    Browse the repository at this point in the history

Commits on Jun 28, 2023

  1. Fix lint issues

    minamijoyo committed Jun 28, 2023
    Configuration menu
    Copy the full SHA
    ae823c8 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    2899273 View commit details
    Browse the repository at this point in the history
  3. Add more debug logs

    minamijoyo committed Jun 28, 2023
    Configuration menu
    Copy the full SHA
    8dd093c View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    e6b979d View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    8a81eea View commit details
    Browse the repository at this point in the history

Commits on Jun 29, 2023

  1. Configuration menu
    Copy the full SHA
    cb4a4bd View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    ba90f8e View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    c05e01e View commit details
    Browse the repository at this point in the history

Commits on Jun 30, 2023

  1. Setup acceptance tests

    minamijoyo committed Jun 30, 2023
    Configuration menu
    Copy the full SHA
    54d666a View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    b0cbd5d View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    4c5160f View commit details
    Browse the repository at this point in the history

Commits on Jul 3, 2023

  1. Fix tests for index

    minamijoyo committed Jul 3, 2023
    Configuration menu
    Copy the full SHA
    7e130e5 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    2f5aa16 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    0efebb6 View commit details
    Browse the repository at this point in the history
  4. Fix typo in testacc

    minamijoyo committed Jul 3, 2023
    Configuration menu
    Copy the full SHA
    f18d544 View commit details
    Browse the repository at this point in the history

Commits on Jul 4, 2023

  1. Fix typo

    minamijoyo committed Jul 4, 2023
    Configuration menu
    Copy the full SHA
    54ab92d View commit details
    Browse the repository at this point in the history