Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

vcp from vospace works when system in readonly mode (CADC 13277) #218

Merged
merged 5 commits into from
Mar 28, 2024
Merged

vcp from vospace works when system in readonly mode (CADC 13277) #218

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
b6ba7b5
Initial version
Mar 26, 2024
6131ffb
Added support for off line and legacy cutouts
Mar 27, 2024
ea8b0af
Fixed unit test
Mar 28, 2024
2489f5b
Fixed typo
Mar 28, 2024
34f45f9
Streamlined cavern calls
Mar 28, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 1 addition & 10 deletions README.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,6 @@ Note: might need to escape chars in your shell
::

cd vos && pip install -e .[test]
cd vofs && pip install -e .[test]

Testing packages
----------------
Expand All @@ -43,14 +42,6 @@ Testing vos
cd ./vos
pytest vos

Testing vofs
~~~~~~~~~~~~

::

cd ./vofs
pytest vofs



Checkstyle
Expand All @@ -60,7 +51,7 @@ not report errors

::

flake8 vos/vos vofs/vofs
flake8 vos/vos


Testing with tox
Expand Down
14 changes: 1 addition & 13 deletions vos/test/scripts/vospace-link-atest.tcsh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -141,20 +141,8 @@ foreach resource ($resources)
echo " [OK]"
endif

echo -n "copy external link"
if ( ${?TESTING_CAVERN} ) then
echo " [SKIPPED, vos/issues/83]"
else
rm -f /tmp/e3link
$CPCMD $CERT -L $CONTAINER/e3link /tmp/ >& /dev/null || echo " [FAIL]" && exit -1
grep -q google /tmp/e3link || echo " [FAIL]" && exit -1
rm -f /tmp/e3link
echo " [OK]"
endif


echo -n "Follow the invalid link and fail"
$CPCMD $CERT $CONTAINER/e3link/somefile /tmp >& /dev/null && echo " [FAIL]" && exit -1
$CPCMD $CERT $CONTAINER/e3link/somefile /tmp >& /dev/null && echo " [FAIL]" && exit -
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the 1 removed on purpose?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The file looks fine to me. It think it's a diff issue maybe

echo " [OK]"

echo -n "copy file to target through link"
Expand Down
8 changes: 3 additions & 5 deletions vos/test/scripts/vospace-lock-atest.tcsh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -121,12 +121,10 @@ foreach resource ($resources)
if ( ${?TESTING_CAVERN} ) then
echo " [SKIPPED, vos/issues/82]"
else
echo "$LOCKCMD $CERT $CONTAINER $VUNLOCK_ARGS"
$LOCKCMD $CERT $CONTAINER $VUNLOCK_ARGS > /dev/null || echo " [FAIL]" && exit -1
echo " [OK]"
echo -n "check unlocked container "
echo "$TAGCMD $CERT $CONTAINER $LIST_ARGS"
$TAGCMD $CERT $CONTAINER $LIST_ARGS | grep -q false && set SUCCESS = "true"
$TAGCMD $CERT $CONTAINER $LIST_ARGS | grep -q None && set SUCCESS = "true"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The grep -q false changed to grep -q None in the $TAGCMD executions, but the grep -q true stayed the same in other execution checks. Is that consistent with the service behaviour?


if ( ${SUCCESS} == "true" ) then
set SUCCESS = "false"
Expand Down Expand Up @@ -175,7 +173,7 @@ foreach resource ($resources)
$LOCKCMD $CERT $CONTAINER/target $VUNLOCK_ARGS> /dev/null || echo " [FAIL]" && exit -1
echo " [OK]"
echo -n "check unlocked link "
$TAGCMD $CERT $CONTAINER/target $LIST_ARGS | grep -q false && set SUCCESS = "true"
$TAGCMD $CERT $CONTAINER/target $LIST_ARGS | grep -q None && set SUCCESS = "true"

if ( ${SUCCESS} == "true" ) then
set SUCCESS = "false"
Expand Down Expand Up @@ -226,7 +224,7 @@ foreach resource ($resources)
$LOCKCMD $CERT $CONTAINER/something.png $VUNLOCK_ARGS> /dev/null || echo " [FAIL]" && exit -1
echo " [OK]"
echo -n "check unlocked node "
$TAGCMD $CERT $CONTAINER/something.png $LIST_ARGS | grep -q false && set SUCCESS = "true"
$TAGCMD $CERT $CONTAINER/something.png $LIST_ARGS | grep -q None && set SUCCESS = "true"
if ( ${SUCCESS} == "true" ) then
set SUCCESS = "false"
echo " [OK]"
Expand Down
29 changes: 14 additions & 15 deletions vos/test/scripts/vospace-node-properties.tcsh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ foreach resource ($resources)
set TIMESTAMP=`date +%Y-%m-%dT%H-%M-%S`
set CONTAINER = $BASE/$TIMESTAMP

echo -n "** checking base URI"
# echo -n "** checking base URI"
# $RMCMD -R $CERT $BASE > /dev/null
# echo -n ", creating base URI"
# $MKDIRCMD $CERT $BASE || echo " [FAIL]" && exit -1
Expand Down Expand Up @@ -107,24 +107,23 @@ foreach resource ($resources)
$LSCMD $CERT $CONTAINER | grep ccc | grep -q "drw-------" || echo " [FAIL]" && exit -1
echo " [OK]"

# echo -n "test vchmod with recursive option"
#
# $CHMODCMD $CERT -R g+r $CONTAINER $GROUP || echo " [FAIL]" && exit -1
# echo -n " verify "
# $LSCMD $CERT $BASE | grep $TIMESTAMP | grep $GROUP | grep -q "drw-r-----" || echo " [FAIL]" && exit -1
# $LSCMD $CERT $CONTAINER | grep aaa | grep $GROUP | grep -q "drw-r-----" || echo " [FAIL]" && exit -1
# $LSCMD $CERT $CONTAINER | grep ccc | grep $GROUP | grep -q "drw-r-----" || echo " [FAIL]" && exit -1
# $LSCMD $CERT $CONTAINER/aaa | grep bbb | grep $GROUP | grep -q "drw-r-----" || echo " [FAIL]" && exit -1
# echo " [OK]"
echo -n "test vchmod with recursive option"

$CHMODCMD $CERT -R g+r $CONTAINER $GROUP || echo " [FAIL]" && exit -1
echo -n " verify "
$LSCMD $CERT $BASE | grep $TIMESTAMP | grep $GROUP | grep -q "drw-r-----" || echo " [FAIL]" && exit -1
$LSCMD $CERT $CONTAINER | grep aaa | grep $GROUP | grep -q "drw-r-----" || echo " [FAIL]" && exit -1
$LSCMD $CERT $CONTAINER | grep ccc | grep $GROUP | grep -q "drw-r-----" || echo " [FAIL]" && exit -1
$LSCMD $CERT $CONTAINER/aaa | grep bbb | grep $GROUP | grep -q "drw-r-----" || echo " [FAIL]" && exit -1
echo " [OK]"

echo -n "test vchmod with multiple groups"

set MULTIGROUP = "ABC $GROUP"
$CHMODCMD $CERT g+r $CONTAINER/aaa "$MULTIGROUP" || echo " [FAIL]" && exit -1
$CHMODCMD $CERT g+r $CONTAINER/aaa "$MULTIGROUP" || echo " [FAIL1]" && exit -1
echo -n " verify "
$LSCMD $CERT $BASE | grep $TIMESTAMP | grep $GROUP | grep -q "drw-r-----" || echo " [FAIL]" && exit -1
echo ""
$LSCMD $CERT $CONTAINER | grep aaa | grep "$MULTIGROUP" | grep -q "drw-r-----" || echo " [FAIL]" && exit -1
$LSCMD $CERT $BASE | grep $TIMESTAMP | grep $GROUP | grep -q "drw-r-----" || echo " [FAIL2]" && exit -1
$LSCMD $CERT $CONTAINER | grep aaa | grep "$MULTIGROUP" | grep -q "drw-r-----" || echo " [FAIL3]" && exit -1
echo " [OK]"

echo -n "make a sub-container public"
Expand All @@ -134,7 +133,7 @@ foreach resource ($resources)
$LSCMD $CERT $BASE | grep $TIMESTAMP | grep $GROUP | grep -q "drw-r-----" || echo " [FAIL1]" && exit -1
$LSCMD $CERT $CONTAINER | grep aaa | grep "$MULTIGROUP" | grep -q "drw-r-----" || echo " [FAIL2]" && exit -1
$LSCMD $CERT $CONTAINER | grep ccc | grep $GROUP | grep -q "drw-r-----" || echo " [FAIL3]" && exit -1
$LSCMD $CERT $CONTAINER/aaa | grep bbb | grep "$MULTIGROUP" | grep -q "drw-r--r--" || echo " [FAIL4]" && exit -1
$LSCMD $CERT $CONTAINER/aaa | grep bbb | grep "$GROUP" | grep -q "drw-r--r--" || echo " [FAIL4]" && exit -1
echo " [OK]"

# echo -n "recursively make all directories public"
Expand Down
13 changes: 7 additions & 6 deletions vos/test/scripts_old/vospace-client-atest.tcsh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -78,9 +78,9 @@ foreach resource ($resources)
echo " [OK]"
endif
echo -n "** setting home and base to public, no groups"
$CHMODCMD $CERT o+r $VOHOME || echo " [FAIL]" && exit -1
$CHMODCMD $CERT o+r $VOHOME || echo " [FAIL1]" && exit -1
echo -n " [OK]"
$CHMODCMD $CERT o+r $BASE || echo " [FAIL]" && exit -1
$CHMODCMD $CERT o+r $BASE || echo " [FAIL2]" && exit -1
echo " [OK]"
echo
echo "*** starting test sequence ***"
Expand Down Expand Up @@ -187,14 +187,15 @@ foreach resource ($resources)
echo " [OK]"

echo -n "copy data node to local filesystem "
$CPCMD $CERT $CONTAINER/something.png $THIS_DIR/something.png.2 || echo " [FAIL]" && exit -1
cmp $THIS_DIR/something.png $THIS_DIR/something.png.2 || echo " [FAIL]" && exit -1

$CPCMD $CERT $CONTAINER/something.png $THIS_DIR/something.png.2 || echo " [FAIL1]" && exit -1
cmp $THIS_DIR/something.png $THIS_DIR/something.png.2 || echo " [FAIL2]" && exit -1
\rm -f $THIS_DIR/something.png.2
echo " [OK]"

echo -n "Check quick copy"
$CPCMD $CERT --quick $THIS_DIR/something.png $CONTAINER/something.png.3|| echo " [FAIL]" && exit -1
$CPCMD $CERT --quick $CONTAINER/something.png.3 $THIS_DIR/something.png.3 || echo " [FAIL]" && exit -1
$CPCMD $CERT --quick $THIS_DIR/something.png $CONTAINER/something.png.3|| echo " [FAIL1]" && exit -1
$CPCMD $CERT --quick $CONTAINER/something.png.3 $THIS_DIR/something.png.3 || echo " [FAIL2]" && exit -1
cmp $THIS_DIR/something.png $THIS_DIR/something.png.3 || echo " [FAIL]" && exit -1
\rm -f $THIS_DIR/something.png.3
echo " [OK]"
Expand Down
12 changes: 0 additions & 12 deletions vos/test/scripts_old/vospace-link-atest.tcsh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -146,18 +146,6 @@ foreach resource ($resources)
echo " [OK]"
endif

echo -n "copy external link"
if ( ${?TESTING_CAVERN} ) then
echo " [SKIPPED, vos/issues/83]"
else
rm -f /tmp/e3link
$CPCMD $CERT -L $CONTAINER/e3link /tmp/ >& /dev/null || echo " [FAIL]" && exit -1
grep -q google /tmp/e3link || echo " [FAIL]" && exit -1
rm -f /tmp/e3link
echo " [OK]"
endif


echo -n "Follow the invalid link and fail"
$CPCMD $CERT $CONTAINER/e3link/somefile /tmp >& /dev/null && echo " [FAIL]" && exit -1
echo " [OK]"
Expand Down
1 change: 1 addition & 0 deletions vos/test/scripts_old/vospace-lock-atest.tcsh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -133,6 +133,7 @@ foreach resource ($resources)
$LOCKCMD $CERT $CONTAINER $VUNLOCK_ARGS > /dev/null || echo " [FAIL]" && exit -1
echo " [OK]"
echo -n "check unlocked container "
echo "$TAGCMD $CERT $CONTAINER $LIST_ARGS"
$TAGCMD $CERT $CONTAINER $LIST_ARGS | grep -q None && set SUCCESS = "true"

if ( ${SUCCESS} == "true" ) then
Expand Down
1 change: 1 addition & 0 deletions vos/test/scripts_old/vospace-node-properties.tcsh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -120,6 +120,7 @@ foreach resource ($resources)
echo " [FAIL]" && exit -1
endif
else
echo "$CHMODCMD $CERT -R g+r $CONTAINER $GROUP1"
$CHMODCMD $CERT -R g+r $CONTAINER $GROUP1 || echo " [FAIL]" && exit -1
echo -n " verify "
$LSCMD $CERT $BASE | grep $TIMESTAMP | grep $GROUP1 | grep -q "drw-r-----" || echo " [FAIL]" && exit -1
Expand Down
4 changes: 2 additions & 2 deletions vos/vos/commands/vcp.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -140,7 +140,7 @@ class Nonlocal():
help="DEPRECATED")
parser.add_argument(
"--quick", action="store_true",
help="assuming CANFAR VOSpace, only comptible with CANFAR VOSpace.",
help="DEPRECATED",
default=False)
parser.add_argument(
"-L", "--follow-links",
Expand Down Expand Up @@ -169,7 +169,7 @@ class Nonlocal():

client = vos.Client(
vospace_certfile=args.certfile, vospace_token=args.token,
transfer_shortcut=args.quick, insecure=args.insecure)
insecure=args.insecure)

if not client.is_remote_file(dest):
dest = os.path.abspath(dest)
Expand Down
Loading
Loading