Refactor current tests and start to use `assert_cmd` #24

mssun · 2018-07-12T23:35:11Z

This PR will refactor current test cases and change to assert_cmd, predicates, assert_fs, etc.

codecov · 2018-07-13T00:00:39Z

Codecov Report

❗ No coverage uploaded for pull request base (master@d47ad31). Click here to learn what that means.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master      #24   +/-   ##
=========================================
  Coverage          ?   13.56%           
=========================================
  Files             ?       46           
  Lines             ?     4349           
  Branches          ?     1705           
=========================================
  Hits              ?      590           
  Misses            ?     3167           
  Partials          ?      592

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d47ad31...0c9293a. Read the comment docs.

…ures directory

The assert_cli is said to be deprecated: assert-rs/assert_cli#41

mssun · 2018-07-15T06:37:45Z

Blocking on a feature of assert_cmd (Feature request: fixture file as input to pipe (with_stdin) assert-rs/assert_cmd#24) to rewrite test cases of head.
Considering to rewrite/reorganize chmod testcases using assert_fs.

epage · 2018-07-16T15:22:58Z

Blocking on a feature of assert_cmd (assert-rs/assert_cmd#24) to rewrite test cases of head.

Do you need to wait for that feature? The alternative is just to read the file and pass it in. The main benefits of that assert_cmd feature are (1) large file support and (2) convenience.

mssun · 2018-07-16T17:19:49Z

Do you need to wait for that feature? The alternative is just to read the file and pass it in. The main benefits of that assert_cmd feature are (1) large file support and (2) convenience.

Yes, reading file and pass it in is a solution. I think with the feature provided by assert_cmd, the testcases will looks more clean. (Current test framework supports pipe in a file). After that, I can safely say that assert_cmd have similar functions of current framework and fully rely on assert_cmd in the future.

Anyway, I will work on refactoring other test cases and come back to this when it is ready. @epage Do you think it will take much more time to have this feature done?

epage · 2018-07-16T20:07:51Z

Anyway, I will work on refactoring other test cases and come back to this when it is ready. @epage Do you think it will take much more time to have this feature done?

I probably won't get to it until tomorrow night. If that is too long, feel free to prepare a PR.

epage · 2018-07-20T15:05:38Z

tests/gnu/test_arch.rs

+    new_cmd!()
+        .assert()
+        .success()
+        .stdout("x86_64\n")


We're considering trimming whitespace for you.

Could you post your thoughts on assert-rs/assert_cmd#19

epage · 2018-07-20T15:11:54Z

tests/macros.rs

+#[macro_export]
+macro_rules! pred_str_contains {
+    ($str:expr) => {
+        predicate::str::contains($str).from_utf8()


RE from_utf8

Right now, the only automatic conversions that happen in our API are u8 slice to predicate and str to predicate.

Feel free to open an issue about converting Predicate<str> into Predicate<[u8]>. or any other ideas to simplify the use of the API.

Yes, this one bother me a lot. I think it could work, but realize it doesn't when compile the code. It took me some time to find out the reason because the documentation of assert_cmd and predicates are mixed. I should check with these two docs.

Could you open issues on any documentation problems you came across? We're wanting to make sure this is accessible for people starting a first Rust project to create a CLI. My challenge is the curse of knowledge: I'm too familiar with it to see know what pitfalls a new user will run into.

mssun · 2018-07-31T21:47:23Z

I think I'm done with the refactoring. @Arcterus

Arcterus · 2018-07-31T22:30:10Z

tests/posix/chmod.rs

-
-#[test]
-#[ignore]
-fn test_chmod_reference_file() {


Can you add this back? It should be supported eventually (I just need to figure out how to get it to work with clap).

It's OK, because I cannot test my rewrite right now. So, let's write test cases then when it's ready.

Arcterus · 2018-08-01T21:44:35Z

The tests seem to run noticeably slower now, although this might just be due to fork+exec being slower than spawning a thread. On the other hand, it should be more resilient in the case of programs setting global state (e.g. we might be able to test the cd built-in for posix/sh now).

mssun · 2018-08-01T22:10:59Z

I don't why the first two cases are blocked.

test gnu::arch::test_x86_64 ... test gnu::arch::test_x86_64 has been running for over 60 seconds
test gnu::base32::test_decode ... test gnu::base32::test_decode has been running for over 60 seconds

Also, not sure if it is related with this issue: assert-rs/assert_cmd#6

Arcterus · 2018-08-01T23:08:59Z

Ah that may be related to some extent. Those test cases work fine for me locally so I’m not sure what’s up with Travis.

Mingshen Sun added 2 commits July 12, 2018 16:30

tests/gnu/arch: change to assert_cli as CLI assertion util

47f09f9

tests/gnu/base32: change to assert_cli as CLI assertion util

80d4ed9

mssun mentioned this pull request Jul 12, 2018

Possibility to replace self-maintained test utility with assert_cli #21

Closed

Mingshen Sun added 2 commits July 12, 2018 16:50

tests: use new_cli macro to simplify creating assert command

b0cc44b

tests/gnu/base64: change to assert_cli as CLI assertion util

6061fcd

Mingshen Sun added 3 commits July 12, 2018 17:33

tests: set tests/fixtures/util as current dir for assert cli

7547355

tests/posix/cat: change to assert_cli as CLI assertion util

abdb172

tests: revert setting the default dir but provide a macro to get fixt…

a4d6e33

…ures directory

mssun changed the title ~~[WIP] Refactor current tests and change to assert_cli~~ [WIP] Refactor current tests and change to assert_cmd Jul 13, 2018

mssun mentioned this pull request Jul 13, 2018

with_stdin will take the ownership of command assert-rs/assert_cmd#22

Closed

tests: use assert_cmd and predicates instead of assert_cli

105839f

The assert_cli is said to be deprecated: assert-rs/assert_cli#41

Arcterus mentioned this pull request Jul 18, 2018

Drop actual_path? #33

Open

mssun added 3 commits July 19, 2018 22:15

tests: update assert_cmd and predicates and fix API changes

95b3766

tests/posix/test_head: start to use assert_cmd

a3fb6ec

tests/posix/test_head: forgot to assert success status

b36360d

epage reviewed Jul 20, 2018

View reviewed changes

mssun mentioned this pull request Jul 27, 2018

Converting from Predicate<str> into Predicate<[u8]> assert-rs/assert_cmd#32

Closed

Mingshen Sun added 7 commits July 31, 2018 11:14

tests/posix/chmod: rewrite chmod testcases

f72f8bd

tests/posix/cat: delete unused mod

a31a9ad

tests/posix/sleep: rewrite sleep test cases

ead136e

Merge master changes into tests/refactor

d085c75

tests/posix/echo: rewrite echo with assert_cmd

126b03b

tests/posix/false: rewrite false with assert_cmd

1d28625

tests/posix/true: rewrite true with assert_cmd

2be4106

Mingshen Sun added 3 commits July 31, 2018 12:54

tests/posix/sleep: relax diff time

48fd4cd

tests/posix/sh: rewrite testcases of sh using assert_cmd

e5f8f17

CI: fix coverage script for libmesabox refactor

3de2bf7

mssun changed the title ~~[WIP] Refactor current tests and change to assert_cmd~~ Refactor current tests and start to use assert_cmd Jul 31, 2018

tests: delete tests framework and unused macros

2a54f58

Arcterus reviewed Jul 31, 2018

View reviewed changes

CI: polish and simplify run-cov script

0c9293a

Arcterus merged commit 7c18432 into master Aug 1, 2018

mssun deleted the tests/refactor branch August 1, 2018 23:32

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Refactor current tests and start to use `assert_cmd` #24

Refactor current tests and start to use `assert_cmd` #24

mssun commented Jul 12, 2018 •

edited

Loading

codecov bot commented Jul 13, 2018 •

edited

Loading

mssun commented Jul 15, 2018

epage commented Jul 16, 2018

mssun commented Jul 16, 2018

epage commented Jul 16, 2018

epage Jul 20, 2018

epage Jul 20, 2018

mssun Jul 21, 2018

epage Jul 21, 2018

mssun commented Jul 31, 2018

Arcterus Jul 31, 2018

mssun Jul 31, 2018

Arcterus commented Aug 1, 2018

mssun commented Aug 1, 2018

Arcterus commented Aug 1, 2018

Refactor current tests and start to use assert_cmd #24

Refactor current tests and start to use assert_cmd #24

Conversation

mssun commented Jul 12, 2018 • edited Loading

codecov bot commented Jul 13, 2018 • edited Loading

Codecov Report

mssun commented Jul 15, 2018

epage commented Jul 16, 2018

mssun commented Jul 16, 2018

epage commented Jul 16, 2018

epage Jul 20, 2018

Choose a reason for hiding this comment

epage Jul 20, 2018

Choose a reason for hiding this comment

mssun Jul 21, 2018

Choose a reason for hiding this comment

epage Jul 21, 2018

Choose a reason for hiding this comment

mssun commented Jul 31, 2018

Arcterus Jul 31, 2018

Choose a reason for hiding this comment

mssun Jul 31, 2018

Choose a reason for hiding this comment

Arcterus commented Aug 1, 2018

mssun commented Aug 1, 2018

Arcterus commented Aug 1, 2018

Refactor current tests and start to use `assert_cmd` #24

Refactor current tests and start to use `assert_cmd` #24

mssun commented Jul 12, 2018 •

edited

Loading

codecov bot commented Jul 13, 2018 •

edited

Loading