Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add team specific template generation #1048

Merged
merged 8 commits into from
Mar 4, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 28 additions & 6 deletions docs/DeveloperGenerateTest.md
Original file line number Diff line number Diff line change
Expand Up @@ -174,6 +174,16 @@ Before generating anything make sure you have Ginkgo in place. To install Ginkgo

`go install -mod=mod github.com/onsi/ginkgo/v2/ginkgo`

## Team specific template
Teams can create their own templates according to their needs.
These should utilize the provided `specs.tmpl` by including the following line wherever they want their specs to be. (do not forget the dot!)

`{{ template "specs" . }}`

Everything needed to get started is in the [templates/default](../templates/default) directory.

Please see the provided [recommended](../templates/default/recommended.tmpl) and [barebones](../templates/default/barebones.tmpl) templates.
Copy them and make your own.

## Usage

Expand Down Expand Up @@ -218,21 +228,33 @@ I0622 22:19:41.865837 20923 textspec.go:67] successfully written to /tmp/outli

```

### Generating the ginkgo spec file from an existing text outline file
This will generate the Ginkgo spec in a subdirectory within our tests directory
### Generating a generic Ginkgo spec file from an existing text outline file
This will generate a generic Ginkgo spec in a subdirectory within our tests directory

`./mage GenerateGinkoSpecFromTextOutline <path>/<to>/<outline> <subpath-under-tests>/<filename>.go`
`./mage GenerateGinkgoSpecFromTextOutline <path>/<to>/<outline> <subpath-under-tests>/<filename>.go`

```bash
$ ./mage GenerateGinkoSpecFromTextOutline dummy_test.outline books/books.go
I0622 22:14:22.140583 20356 testcasemapper.go:58] Mapping outline from a text file, dummy_test.outline
I0622 22:14:22.140673 20356 testcasemapper.go:47] Mapping outline to a Ginkgo test file, books/books.go
I0622 22:14:22.140841 20356 ginkgosspec.go:242] Creating new test package directory and spec file tests/books/books.go.


```
As noted above, this command will create a new package under the `tests/` directory and a test spec file `<filename>.go` for you. It will contain some basic imports but more importantly it will generate a basic structured Ginkgo spec skeleton that you can code against.

### Generating a team specific Ginkgo spec file from an existing text outline file
This will generate the Ginkgo spec in a subdirectory within our tests directory using a team specific template provided by user. Please see the [Team specific template](#team-specific-template) section.

Feel free to use the provided [testOutline](../templates/default/testOutline) file for testing.

`./mage GenerateTeamSpecificGinkgoSpecFromTextOutline <path>/<to>/<outline> <path>/<to>/<team-specific-template> <subpath-under-tests>/<filename>.go`

```bash
➜ ./mage GenerateTeamSpecificGinkgoSpecFromTextOutline templates/default/testOutline templates/default/recommended.tmpl tests/template_poc/template_poc.go
I0219 15:42:17.808595 351210 magefile.go:755] Mapping outline from a text file, templates/default/testOutline
I0219 15:42:17.808685 351210 magefile.go:762] Mapping outline to a Ginkgo spec file, tests/template_poc/template_poc.go
I0219 15:42:17.809210 351210 ginkgosspec.go:144] Creating new test package directory and spec file /home/tnevrlka/Work/e2e-tests/tests/template_poc/template_poc.go.
```

### Printing a text outline in JSON format of an existing ginkgo spec file
This will generate the outline and output to your terminal in JSON format. This is the format we use when rendering the template. You can pipe this output to tools like `jq` for formatting and filtering. This would only be useful for troubleshooting purposes

Expand Down Expand Up @@ -298,4 +320,4 @@ $ ./mage GenerateTestSuiteFile chaos
I0623 12:48:13.761038 31196 magefile.go:467] Creating new test suite file cmd/chaos_test.go.
cmd/chaos_test.go

```
```
17 changes: 11 additions & 6 deletions magefiles/magefile.go
Original file line number Diff line number Diff line change
Expand Up @@ -24,12 +24,12 @@ import (
gh "github.com/google/go-github/v44/github"
"github.com/magefile/mage/sh"
"github.com/redhat-appstudio/e2e-tests/magefiles/installation"
"github.com/redhat-appstudio/e2e-tests/magefiles/testspecs"
"github.com/redhat-appstudio/e2e-tests/magefiles/upgrade"
"github.com/redhat-appstudio/e2e-tests/pkg/clients/github"
"github.com/redhat-appstudio/e2e-tests/pkg/clients/slack"
"github.com/redhat-appstudio/e2e-tests/pkg/clients/sprayproxy"
"github.com/redhat-appstudio/e2e-tests/pkg/constants"
"github.com/redhat-appstudio/e2e-tests/pkg/testspecs"
"github.com/redhat-appstudio/e2e-tests/pkg/utils"
"github.com/redhat-appstudio/e2e-tests/pkg/utils/tekton"
"github.com/redhat-appstudio/image-controller/pkg/quay"
Expand Down Expand Up @@ -758,26 +758,31 @@ func GenerateTextOutlineFromGinkgoSpec(source string, destination string) error
}

// Generate a Ginkgo Spec file from a Text Outline file
func GenerateGinkoSpecFromTextOutline(source string, destination string) error {
func GenerateGinkgoSpecFromTextOutline(source string, destination string) error {
return GenerateTeamSpecificGinkgoSpecFromTextOutline(source, testspecs.TestFilePath, destination)
}

// Generate a team specific file using specs in templates/specs.tmpl file and a provided team specific template
func GenerateTeamSpecificGinkgoSpecFromTextOutline(outlinePath, teamTmplPath, destinationPath string) error {
gs := testspecs.NewGinkgoSpecTranslator()
ts := testspecs.NewTextSpecTranslator()

klog.Infof("Mapping outline from a text file, %s", source)
outline, err := ts.FromFile(source)
klog.Infof("Mapping outline from a text file, %s", outlinePath)
outline, err := ts.FromFile(outlinePath)
if err != nil {
klog.Error("Failed to map text outline file")
return err
}

klog.Infof("Mapping outline to a Ginkgo spec file, %s", destination)
err = gs.ToFile(destination, outline)
klog.Infof("Mapping outline to a Ginkgo spec file, %s", destinationPath)
err = gs.ToFile(destinationPath, teamTmplPath, outline)
if err != nil {
klog.Error("Failed to map Ginkgo spec file")
return err
}

return err

}

// Print the outline of the Ginkgo spec
Expand Down
File renamed without changes.
8 changes: 8 additions & 0 deletions pkg/testspecs/const.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
package testspecs

const (
TestFilePath = "templates/test_output_spec.tmpl"
FrameworkDescribePath = "templates/framework_describe_func.tmpl"

SpecsPath = "templates/specs.tmpl"
)
67 changes: 56 additions & 11 deletions magefiles/testspecs/ginkgosspec.go → pkg/testspecs/ginkgosspec.go
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,9 @@ package testspecs
import (
"encoding/json"
"fmt"

"io"
"os"
"path"
"path/filepath"
"reflect"
"strings"
Expand Down Expand Up @@ -54,7 +55,7 @@ func (gst *GinkgosSpecTranslator) FromFile(file string) (TestOutline, error) {
}

// ToFile generates a Ginkgo test file from a TestOutline
func (gst *GinkgosSpecTranslator) ToFile(destination string, outline TestOutline) error {
func (gst *GinkgosSpecTranslator) ToFile(destination, teamTmplPath string, outline TestOutline) error {

e2ePath, err := os.Getwd()
if err != nil {
Expand All @@ -71,7 +72,7 @@ func (gst *GinkgosSpecTranslator) ToFile(destination string, outline TestOutline
return err
}

return generateGinkgoSpec(e2ePath, testFilePath, dataFile)
return generateGinkgoSpec(e2ePath, teamTmplPath, testFilePath, dataFile)

}

Expand Down Expand Up @@ -112,10 +113,19 @@ func excludeSetupTeardownNodes(nodes TestOutline) TestOutline {
// generateGinkgoSpec will call the ginkgo generate command
// to generate the ginkgo data json file we created and
// the template located in out templates directory
func generateGinkgoSpec(cwd string, destination string, dataFile string) error {
func generateGinkgoSpec(cwd, teamTmplPath, destination string, dataFile string) error {

var err error

if teamTmplPath != TestFilePath {
tmplFile, err := mergeTemplates(teamTmplPath, SpecsPath)
if err != nil {
return err
}
defer os.Remove(tmplFile.Name())
teamTmplPath = tmplFile.Name()
}

// Note I change into the directory and rename things because ginkgo
// by default generates the test file name as <package>_test.go.
// Since that is not a semantic we follow I perform this action.
Expand All @@ -134,15 +144,9 @@ func generateGinkgoSpec(cwd string, destination string, dataFile string) error {
// Doing this to avoid errcheck flagging this in a defer.
// Refer to https://github.com/kisielk/errcheck
// issues 101, 77, 55
tmpl, err := GetTemplate("test-file")
if err != nil {
return err
}

fullTemplatePath := fmt.Sprintf("%s/%s", cwd, tmpl)

klog.Infof("Creating new test package directory and spec file %s.\n", destination)
_, err = ginkgoGenerateSpecCmd("--template", fullTemplatePath, "--template-data", dataFile)
_, err = ginkgoGenerateSpecCmd("--template", teamTmplPath, "--template-data", dataFile)
if err != nil {
err = os.Remove(ginkgoFileName)
if err != nil {
Expand Down Expand Up @@ -170,6 +174,47 @@ func generateGinkgoSpec(cwd string, destination string, dataFile string) error {
return err
}

// mergeTemplates creates a new template file from files provided in the argument
func mergeTemplates(paths ...string) (*os.File, error) {
cwd, err := os.Getwd()
if err != nil {
return nil, err
}

tempFile, err := os.CreateTemp(cwd, "merged-tmpl")
if err != nil {
return nil, err
}
defer tempFile.Close()

for _, templatePath := range paths {
// Avoid possible memory leak caused by defer by wrapping in a function
appendToTempPath := func() error {
tmplFile, err := os.Open(path.Clean(templatePath))
if err != nil {
return err
}
defer tmplFile.Close()

_, err = io.Copy(tempFile, tmplFile)
if err != nil {
return err
}

_, err = tempFile.Write([]byte{'\n', '\n'})
if err != nil {
return err
}
return nil
}
err = appendToTempPath()
if err != nil {
return nil, fmt.Errorf("error during appending to temp templatePath: %+v", err)
}
}
return tempFile, nil
}

// createTestPath will create the full test path in the tests
// directory if it doesn't exit
func createTestPath(cwd string, destination string) (string, error) {
Expand Down
69 changes: 69 additions & 0 deletions pkg/testspecs/ginkgosspec_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
package testspecs

import (
"fmt"
"os"
"testing"
)

func TestMergeTemplates(t *testing.T) {
cwd, err := os.Getwd()
if err != nil {
t.Fatal(err)
return
}

tempDir, err := os.MkdirTemp(cwd, "test-merge-templates")
if err != nil {
t.Fatal(err)
return
}
defer os.RemoveAll(tempDir)

err = os.Chdir(tempDir)
if err != nil {
t.Fatal(err)
return
}

var fileNames []string
var expectedString string

for i := 0; i < 10; i++ {
file, err := os.CreateTemp(tempDir, "tempFile")
if err != nil {
t.Fatal(err)
return
}
lineContent := fmt.Sprintf("This should be line number '%d' from '%s'", i, file.Name())
_, err = file.WriteString(lineContent)
if err != nil {
t.Fatal(err)
return
}
expectedString += lineContent + "\n\n"
fileNames = append(fileNames, file.Name())
}

mergedFile, err := mergeTemplates(fileNames...)
if err != nil {
t.Errorf("failed to merge templates: %+v", err)
return
}

mergedFile, err = os.Open(mergedFile.Name())
if err != nil {
t.Error(err)
return
}
mergedFileBytes, err := os.ReadFile(mergedFile.Name())
if err != nil {
t.Error(err)
return
}

mergedFileContent := string(mergedFileBytes)
if mergedFileContent != expectedString {
t.Errorf("content of merged file does not match the expected content")
}
}
19 changes: 1 addition & 18 deletions magefiles/testspecs/templates.go → pkg/testspecs/templates.go
Original file line number Diff line number Diff line change
Expand Up @@ -13,11 +13,6 @@ import (
"k8s.io/klog/v2"
)

var templates = map[string]string{
"test-file": "templates/test_output_spec.tmpl",
"framework-describe": "templates/framework_describe_func.tmpl",
}

func NewTemplateData(specOutline TestOutline, destination string) *TemplateData {

// This regex will find all the words that start with capital letter
Expand Down Expand Up @@ -47,22 +42,10 @@ func NewTemplateData(specOutline TestOutline, destination string) *TemplateData
return &TemplateData{Outline: specOutline, PackageName: dirName, FrameworkDescribeString: newSpecName}
}

func GetTemplate(name string) (string, error) {
if s, ok := templates[name]; ok {
return s, nil
}
return "", fmt.Errorf("no Template found for %q", name)
}

func RenderFrameworkDescribeGoFile(t TemplateData) error {

templatePath, err := GetTemplate("framework-describe")
if err != nil {
return err
}
var describeFile = "pkg/framework/describe.go"

err = renderTemplate(describeFile, templatePath, t, true)
err := renderTemplate(describeFile, FrameworkDescribePath, t, true)
if err != nil {
klog.Errorf("failed to append to pkg/framework/describe.go with : %s", err)
return err
Expand Down
File renamed without changes.
File renamed without changes.
45 changes: 45 additions & 0 deletions templates/default/recommended.tmpl
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
package {{ .CustomData.PackageName }}

/* This was generated from a template file. Please feel free to update as necessary!
a couple things to note:
- Remember to implement specific logic of the service/domain you are trying to test if it not already there in the pkg/

- To include the tests as part of the E2E Test suite:
- Update the pkg/framework/describe.go to include the `Describe func` of this new test suite, If you haven't already done so.
- Import this new package into the cmd/e2e_test.go
*/

import (
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"

"fmt"
"strings"
"time"
"encoding/json"
"context"


"github.com/redhat-appstudio/e2e-tests/pkg/framework"
//framework imports edit as required
"github.com/redhat-appstudio/e2e-tests/pkg/constants"
"github.com/redhat-appstudio/e2e-tests/pkg/utils"

)

{{ range .CustomData.Outline }}
var _ = framework.{{ .Name }}("{{ .Text }}", {{range .Labels }}Label("{{.}}"), {{ end }} func() {
defer GinkgoRecover()
var err error
var f *framework.Framework
// use 'f' to access common controllers or the specific service controllers within the framework
BeforeAll(func() {
// Initialize the tests controllers
f, err = framework.NewFramework()
Expect(err).NotTo(HaveOccurred())
})

// Generated specs:
{{ template "specs" . }}
})
{{ end }}
Loading
Loading