error when useing ceiba-dl get command

[Wilson1211]

$ ceiba-dl get
Traceback (most recent call last):
File "/usr/local/bin/ceiba-dl", line 243, in
exit(0 if args.func(args, config) else 1)
File "/usr/local/bin/ceiba-dl", line 141, in run_get
end_download_callback=end_callback)
File "/usr/local/lib/python3.5/site-packages/ceiba_dl/init.py", line 388, in run
download_progress_callback, end_download_callback)
File "/usr/local/lib/python3.5/site-packages/ceiba_dl/init.py", line 232, in download_file
if not self.download_file(child_path, retry, dcb, ecb):
File "/usr/local/lib/python3.5/site-packages/ceiba_dl/init.py", line 232, in download_file
if not self.download_file(child_path, retry, dcb, ecb):
File "/usr/local/lib/python3.5/site-packages/ceiba_dl/init.py", line 232, in download_file
if not self.download_file(child_path, retry, dcb, ecb):
File "/usr/local/lib/python3.5/site-packages/ceiba_dl/init.py", line 214, in download_file
node = self.vfs.open(path)
File "/usr/local/lib/python3.5/site-packages/ceiba_dl/vfs.py", line 40, in open
work.fetch()
File "/usr/local/lib/python3.5/site-packages/ceiba_dl/vfs.py", line 979, in fetch
course_list_row = self.vfs.root.courses.search_course_list(self._sn)
File "/usr/local/lib/python3.5/site-packages/ceiba_dl/vfs.py", line 397, in search_course_list
return self._course_list_map[sn]
KeyError: '0ff983'

[lantw44]

It seems that CEIBA API returns a course which does not exist on https://ceiba.ntu.edu.tw/student/index.php page. 0ff983 is the CEIBA serial number (csn) of the problematic course.

[Wilson1211]

It seems that courses in 旁聽 have the error described above. Is there any approach to bypass the problem?

[lantw44]

It seems that courses in 旁聽 have the error described above.

Since I have no course marked as 旁聽 in my account, I can't reproduce the issue myself. Does this patch work for you?

diff --git a/ceiba_dl/vfs.py b/ceiba_dl/vfs.py
index a5c6c59..56b6c22 100644
--- a/ceiba_dl/vfs.py
+++ b/ceiba_dl/vfs.py
@@ -394,6 +394,8 @@ class RootCoursesDirectory(Directory):
     def search_course_list(self, sn):
         if not hasattr(self, '_course_list_map'):
             self._create_course_list_map()
+        if sn not in self._course_list_map:
+            return []
         return self._course_list_map[sn]
 
 class RootStudentsDirectory(Directory):

Is there any approach to bypass the problem?

You can add the broken directory to the delete_files list in edit section of your configuration file.

[Wilson1211]

It shows the error message:

Traceback (most recent call last):
File "/usr/local/bin/ceiba-dl", line 243, in
exit(0 if args.func(args, config) else 1)
File "/usr/local/bin/ceiba-dl", line 141, in run_get
end_download_callback=end_callback)
File "/usr/local/lib/python3.5/site-packages/ceiba_dl/init.py", line 388, in run
download_progress_callback, end_download_callback)
File "/usr/local/lib/python3.5/site-packages/ceiba_dl/init.py", line 214, in download_file
node = self.vfs.open(path)
File "/usr/local/lib/python3.5/site-packages/ceiba_dl/vfs.py", line 40, in open
work.fetch()
File "/usr/local/lib/python3.5/site-packages/ceiba_dl/vfs.py", line 982, in fetch
if len(course_list_row[6]) > 0:
IndexError: list index out of range

[lantw44]

I am not sure if this patch can work around your problem:

diff --git a/ceiba_dl/vfs.py b/ceiba_dl/vfs.py
index a5c6c59..f042aaf 100644
--- a/ceiba_dl/vfs.py
+++ b/ceiba_dl/vfs.py
@@ -394,6 +394,8 @@ class RootCoursesDirectory(Directory):
     def search_course_list(self, sn):
         if not hasattr(self, '_course_list_map'):
             self._create_course_list_map()
+        if sn not in self._course_list_map:
+            return [[]] * 8
         return self._course_list_map[sn]
 
 class RootStudentsDirectory(Directory):

Since I have no access to your data, it is unlikely that I am able to come up with a working patch for you. The entire project is built on undocumented interfaces of NTU CEIBA, so it is impossible for me to make it reliable. Therefore, when you see an issue, it is recommended to debug and fix it yourself and submit the patch after you resolve the issue. If you can't fix it, please use delete_files to ignore the file or the directory causing the issue.

[b04505009]

This works for me since 旁聽 courses are just listed on table 2

diff --git a/ceiba_dl/vfs.py b/ceiba_dl/vfs.py
old mode 100644
new mode 100755
index a5c6c59..d5b8ec7
--- a/ceiba_dl/vfs.py
+++ b/ceiba_dl/vfs.py
@@ -353,6 +353,7 @@ class RootCoursesDirectory(Directory):
         course_list_page = self.vfs.request.web('/student/index.php')
         course_list_rows_all = course_list_page.xpath('//table[1]/tr')
         course_list_rows = course_list_rows_all[1:]
+        course_list_rows += course_list_page.xpath('//table[2]/tr')[1:]
         course_list_header_row = course_list_rows_all[0]

[lantw44]

This works for me since 旁聽 courses are just listed on table 2

Sorry, it doesn't work for me because I have no 旁聽 courses, and the second table has the title '以下為您修習的課程，並未於CEIBA上開設課程網頁' instead, which causes _create_course_list_map to fail because it can't find the CEIBA serial number.

Traceback (most recent call last):
  File "/home/lantw44/devel/ceiba-dl/ceiba-dl", line 243, in <module>
    exit(0 if args.func(args, config) else 1)
  File "/home/lantw44/devel/ceiba-dl/ceiba-dl", line 158, in run_ls
    lser.run(sys.stdout, path)
  File "/home/lantw44/devel/ceiba-dl/ceiba_dl/__init__.py", line 433, in run
    self.print_file(output, path, self.recursive)
  File "/home/lantw44/devel/ceiba-dl/ceiba_dl/__init__.py", line 397, in print_file
    node = self.vfs.open(path)
  File "/home/lantw44/devel/ceiba-dl/ceiba_dl/vfs.py", line 27, in open
    self._do_edit()
  File "/home/lantw44/devel/ceiba-dl/ceiba_dl/vfs.py", line 63, in _do_edit
    course.fetch()
  File "/home/lantw44/devel/ceiba-dl/ceiba_dl/vfs.py", line 1089, in fetch
    course_list_row = self.vfs.root.courses.search_course_list(self._sn)
  File "/home/lantw44/devel/ceiba-dl/ceiba_dl/vfs.py", line 397, in search_course_list
    self._create_course_list_map()
  File "/home/lantw44/devel/ceiba-dl/ceiba_dl/vfs.py", line 393, in _create_course_list_map
    assert False
AssertionError