Bugfixes and new rules from collaboration EUMETSAT-AQCLab #230

jverdugo-aqclab · 2022-12-21T12:03:06Z

Proposed changes

Fixing of parsing errors detected in three rules: COM.Inst.Brace, COM.PROJECT.Header, COM.NAME.Homonymy

Implementation of 8 new rules (from EUMETSAT coding standard, but useful in other contexts) to check:

size of files (lines of code) againts a threshold: The rule checks if a program or module exceeds a maximum number of lines of code (excluding comments and blank lines) set at 1000 by default.
size of procedures (lines of code) againts a threshold: The rule checks if a procedure (function or subroutine) exceeds a maximum number of lines of code (excluding comments and blank lines) set at 150 by default.
percentage of comments againts a threshold: The rule checks if a program or module has at least a certain percentage of comments set at 30% by default.
cyclomatic complexity of procedures againts a threshold: The rule checks if a procedure (function or subroutine) exceeds a maximum number of cyclomatic complexity (simplified, not taking goto statements into account) set at 15 by default.
declaration of only one program/module per file: Each file should only contain one program or module. The rule checks if there are several of them in one file.
number of arguments for procedures againts a threshold: The rule checks if a procedure (function or subroutine) has more than a maximum number of arguments or parameters, set at 7 by default.
documentation of variables: The rule checks that the declaration of variables and types is preceded by a comment line that describes its use.
content in headers: The rule checks that headers for modules or programs have, at least, the following components: Unique identification of the program or module, name of the file, author’s name, statement of the Copyright ownership of the code, description of the nature of functions/subroutines in the file

Types of changes

What types of changes does your code introduce to i-Code?

Bugfix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to not work as expected)
Documentation Update (if none of the other choices apply)

Issues closed by changes

Checklist

I have read the CONTRIBUTING doc
I agree with the CODE OF CONDUCT
Lint and unit tests pass locally with my changes
SonarCloud and GitHub Actions tests pass with my changes
I have added tests that prove my fix is effective or that my feature works
I have added necessary documentation (if appropriate)
Any dependent changes have been merged and published in downstream modules

Further comments

The changes implemented were sponsored and agreed with EUMETSAT, based on their needs, but both EUMETSAT and AQCLab consider that these may be useful to other parties, and for that may be considered to be included in the offical development of i-Code.

…ndexOutOfBoundsException) were produced when analyzed files contained declaration of functions or subroutines with modifiers Recursive, Elemental or Pure

…a file only contains comments, no source code. Also a JFlexException was thrown in the case of a file without code nor comments. The throw for this Exception has been removed to avoid parsing errors in the results.

…n analyzing files that included sentences like "PROCEDURE(proc_interface), POINTER :: proc"

…ld of LOC (set at 1000). Only lines of code are taken into account, comments and blank lines are ignored.

…eshold of LOC (set at 150). Only lines of code are taken into account, comments and blank lines are ignored.

Rule to check the maximum number of arguments in each procedure, which must be at most 7.

Rule to check the maximum value of the cyclomatic complexity in each function, which at most must be 15.

Rule to check that only one module or program is declared in each file.

Rule to check that in each file there are at least 30% of lines of comments with respect to the total number of lines.

diegorodriguez31

Please can you provide me some Fortran files that break the new rules and the fixed rules, so that I can test it?
The merge request will be approved after the new rules tested and when my suggestions will be taken into account or discussed with me to clarify.

fortran77-rules/src/main/resources/lex/COMFLOWCheckArguments.lex

fortran77-rules/src/main/resources/lex/COMMETComplexitySimplified.lex

fortran77-rules/src/main/resources/lex/COMMETLineOfCode.lex

fortran90-rules/src/main/resources/lex/COMFLOWCheckArguments.lex

diegorodriguez31 · 2023-03-28T13:03:04Z

fortran90-rules/src/main/resources/fortran90-rules.xml

+        <languageId>fr.cnes.icode.fortran90</languageId>
+    </check>
+    <check>


Remove it if you decide to enhance the existing header rule

Pending of confirmation on comment #230 (comment)

diegorodriguez31 · 2023-03-28T13:03:23Z

fortran77-rules/src/main/resources/fortran77-rules.xml

+        <languageId>fr.cnes.icode.fortran77</languageId>
+    </check>
+    <check>
+        <class>fr.cnes.icode.fortran77.rules.F77FILEHeader</class>


Remove it if you decide to enhance the existing header rule

Pending of confirmation on comment #230 (comment)

diegorodriguez31 · 2023-03-28T13:06:02Z

fortran77-rules/src/main/resources/lex/COMPROJECTHeader.lex

@@ -109,47 +109,43 @@ STRING		 = \'[^\']*\' | \"[^\"]*\"

 	private void raiseErrors() throws JFlexException {	
        LOGGER.finest("begin method raiseErrors");
-		if(linesType.isEmpty()){


You decided to remove the fact that we raise an error if file is blank.
Can it be possible to really have blank files in FORTRAN?
If yes, then I will join your choice.
If no, we should implement a rule checking if there is empty files.

Raising an error in this specific rule and not in others seemed arbitrary, since checking empty files is not the purpose of this rule.
Having a blank file is possible (we found it in one of the projects we evaluated at EUMETSAT). Even though it doesn't present a great problem, such a file would be useless and should be avoided, the same way unused variables should be avoided. So, for this purpose we agree on having a specific rule that checks this situation.
As an alternative, we would favor simply ignoring blank files, but not raise an error in an unrelated rule such as COMPROJECTHeader.

diegorodriguez31 · 2023-03-28T13:06:55Z

fortran90-rules/src/main/resources/lex/COMPROJECTHeader.lex

@@ -104,90 +104,86 @@ STRING		 = \'[^\']*\' | \"[^\"]*\"
 	}

 	private void raiseErrors() throws JFlexException {
-		if(this.linesType.isEmpty()){


Same comment than for fortran77 COMPROJECTHeader

Raising an error in this specific rule and not in others seemed arbitrary, since checking empty files is not the purpose of this rule.
Having a blank file is possible (we found it in one of the projects we evaluated at EUMETSAT). Even though it doesn't present a great problem, such a file would be useless and should be avoided, the same way unused variables should be avoided. So, for this purpose we agree on having a specific rule that checks this situation.
As an alternative, we would favor simply ignoring blank files, but not raise an error in an unrelated rule such as COMPROJECTHeader.

diegorodriguez31

Change these values too to respect RNC

fortran77-rules/src/main/resources/lex/COMMETRatioComment.lex

fortran90-rules/src/main/resources/lex/COMMETRatioComment.lex

Co-authored-by: Diego Rodriguez <[email protected]>

…ied.lex Co-authored-by: Diego Rodriguez <[email protected]>

Co-authored-by: Diego Rodriguez <[email protected]>

…ied.lex Co-authored-by: Diego Rodriguez <[email protected]>

Co-authored-by: Diego Rodriguez <[email protected]>

jverdugo-aqclab and others added 11 commits December 21, 2022 13:00

Bug fix: parsing errors (COM.NAME.Homonymy: Location unreachable or I…

f06ee24

…ndexOutOfBoundsException) were produced when analyzed files contained declaration of functions or subroutines with modifiers Recursive, Elemental or Pure

Bug fix: a parsing error (IndexOutOfBoundsException) was produced whe…

50108d9

…n analyzing files that included sentences like "PROCEDURE(proc_interface), POINTER :: proc"

Rule for checking that lines of code in a file don't exceed a thresho…

8104472

…ld of LOC (set at 1000). Only lines of code are taken into account, comments and blank lines are ignored.

Rule for checking that lines of code in procedures don't exceed a thr…

8178b61

…eshold of LOC (set at 150). Only lines of code are taken into account, comments and blank lines are ignored.

Rule ArgumentsProcedure

53bcf92

Rule to check the maximum number of arguments in each procedure, which must be at most 7.

Rule CyclomaticComplexity

fd38d56

Rule to check the maximum value of the cyclomatic complexity in each function, which at most must be 15.

Rule LogicUnit

a35a601

Rule to check that only one module or program is declared in each file.

Rule PercentageComment

f3fe830

Rule to check that in each file there are at least 30% of lines of comments with respect to the total number of lines.

Rule CommentVar

363f560

Rule Header

fd9331d

jverdugo-aqclab changed the title ~~Eumetsat aqc lab 2~~ Bugfixes and new rules from collaboration EUMETSAT-AQCLab Dec 22, 2022

Diego added 2 commits March 28, 2023 13:30

Change new rules names to RNC rules

c73b574

Change name in comment

1b2592e

diegorodriguez31 reviewed Mar 28, 2023

View reviewed changes

diegorodriguez31 reviewed Mar 29, 2023

View reviewed changes

jverdugo-aqclab and others added 14 commits June 2, 2023 11:27

Update fortran77-rules/src/main/resources/lex/COMFLOWCheckArguments.lex

8d81c1e

Co-authored-by: Diego Rodriguez <[email protected]>

Update fortran77-rules/src/main/resources/lex/COMFLOWCheckArguments.lex

081584c

Co-authored-by: Diego Rodriguez <[email protected]>

Update fortran77-rules/src/main/resources/lex/COMMETComplexitySimplif…

93dbf77

…ied.lex Co-authored-by: Diego Rodriguez <[email protected]>

Update fortran77-rules/src/main/resources/lex/COMMETComplexitySimplif…

0ff4252

…ied.lex Co-authored-by: Diego Rodriguez <[email protected]>

Update fortran77-rules/src/main/resources/lex/COMMETLineOfCode.lex

a733ffd

Co-authored-by: Diego Rodriguez <[email protected]>

Update fortran77-rules/src/main/resources/lex/COMMETLineOfCode.lex

446f106

Co-authored-by: Diego Rodriguez <[email protected]>

Update fortran90-rules/src/main/resources/lex/COMMETLineOfCode.lex

3bf3c33

Co-authored-by: Diego Rodriguez <[email protected]>

Update fortran90-rules/src/main/resources/lex/COMFLOWCheckArguments.lex

23ac9c7

Co-authored-by: Diego Rodriguez <[email protected]>

Update fortran77-rules/src/main/resources/lex/COMMETRatioComment.lex

290cd74

Co-authored-by: Diego Rodriguez <[email protected]>

Update fortran77-rules/src/main/resources/lex/COMMETRatioComment.lex

8e43b0c

Co-authored-by: Diego Rodriguez <[email protected]>

Update fortran90-rules/src/main/resources/lex/COMMETRatioComment.lex

dd56e54

Co-authored-by: Diego Rodriguez <[email protected]>

Update fortran90-rules/src/main/resources/lex/COMMETRatioComment.lex

217f47e

Co-authored-by: Diego Rodriguez <[email protected]>

Update fortran90-rules/src/main/resources/lex/COMMETLineOfCode.lex

8502935

Co-authored-by: Diego Rodriguez <[email protected]>

Update fortran90-rules/src/main/resources/lex/COMMETComplexitySimplif…

849784b

…ied.lex Co-authored-by: Diego Rodriguez <[email protected]>

jverdugo-aqclab and others added 5 commits June 2, 2023 11:41

Update fortran90-rules/src/main/resources/lex/COMMETComplexitySimplif…

af8085a

…ied.lex Co-authored-by: Diego Rodriguez <[email protected]>

Update fortran90-rules/src/main/resources/lex/COMFLOWCheckArguments.lex

ed18b31

Co-authored-by: Diego Rodriguez <[email protected]>

Remove unused variable numTotal

a66e884

Header updated to reflect this rule is not defined in RNC.

5e8cac9

Test cases for new rules

8b510ac

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Bugfixes and new rules from collaboration EUMETSAT-AQCLab #230

Bugfixes and new rules from collaboration EUMETSAT-AQCLab #230

jverdugo-aqclab commented Dec 21, 2022 •

edited

Loading

diegorodriguez31 left a comment

diegorodriguez31 Mar 28, 2023

jverdugo-aqclab Jun 5, 2023

diegorodriguez31 Mar 28, 2023

jverdugo-aqclab Jun 5, 2023

diegorodriguez31 Mar 28, 2023

jverdugo-aqclab Jun 5, 2023

diegorodriguez31 Mar 28, 2023

jverdugo-aqclab Jun 5, 2023

diegorodriguez31 left a comment

Bugfixes and new rules from collaboration EUMETSAT-AQCLab #230

Are you sure you want to change the base?

Bugfixes and new rules from collaboration EUMETSAT-AQCLab #230

Conversation

jverdugo-aqclab commented Dec 21, 2022 • edited Loading

Proposed changes

Types of changes

Issues closed by changes

Checklist

Further comments

diegorodriguez31 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

diegorodriguez31 left a comment

Choose a reason for hiding this comment

jverdugo-aqclab commented Dec 21, 2022 •

edited

Loading