ServerRelativeURL not correctly encoded, leading to inability to download filenames with % etc. in

[a-h]

Describe the bug

If I create a file with a '%' symbol in Sharepoint, attempts to retrieve it using GetFile fail.

I suspect it will also fail with ' due to string concatenation being used to create API calls, and & and other URL parameters being used due to invalid path escaping.

time=2024-10-30T12:39:49.085Z level=ERROR msg="failed to get file" file="" error="unable to create a request: parse \"https://sharepoint.example.com/sites/MySite/_api/Web/GetFileByServerRelativeUrl('/sites/MySite/Shared Documents/General/Doc with extra 20% correctness.docx')\": invalid URL escape \"% c\""
exit status 1

Versions

• SharePoint Online.
• github.com/koltyakov/gosip v0.0.0-20240227190045-46a1ea645a98

To Reproduce

• Create Sharepoint site
• Add file with percentage symbol in the name, e.g. After 20% increase.docx
• Use sample app at https://github.com/a-h/spchars to access the site.
• Note error message.

Expected behavior

Since the RelativeServerURL was returned by the API, I expected to be able to pass it to other API calls without issue.

Screenshots

No screenshots.

Additional context

This seems to a problem with how ServerRelativeURL values are returned. For example, when calling sp.Web().GetFolder(*flagFolder).Files().Get() the resultant JSON payload contains a ServerRelativeURL % symbol, without it being properly URL path escaped to %25.

{
  "ServerRelativeUrl": "/sites/MySite/Shared Documents/General/Doc with extra 20% correctness.docx",
}

Within api/web.go there's some string formatting to create files, which looks like it is subject to the equivalent of a SQL injection, in that you can add a quote into the serverRelativeURL and it would perhaps break?

func (web *Web) GetFile(serverRelativeURL string) *File {
	return NewFile(
		web.client,
		fmt.Sprintf(
			"%s/GetFileByServerRelativeUrl('%s')",
			web.endpoint,
			checkGetRelativeURL(serverRelativeURL, web.endpoint),
		),
		web.config,
	)
}

This is discussed here SharePoint/sp-dev-docs#6673 (comment)

I don't know enough about Sharepoint to know whether the return value of ServerRelativeURL should be URL encoded or not (maybe not), but it looks like it should be URL encoded during GetFile and similar calls, using url.PathEscape - https://pkg.go.dev/net/url#PathEscape

A workaround is to use the UniqueID to access files and folders. The UniqueID related API calls seem to have been introduced in Sharepoint 2015.

I saw #78 but it was closed, and I thought I should add a reproduction and extra context.

Thanks for your work on this library, it's very helpful!

[koltyakov]

Thanks @a-h, I'll check it out. Yet, it can be a loop of encoding vs not encoding (ending up some names never work), some SharePoint APIs might be historically inconsistent, as a rule I try applying little transformation in the middle to keep authentic API behavior. Yet, let me check for this test case if it can be resolved.

Btw, much appreciation for your contribution in Go, like templ project. 🚀

[koltyakov]

So far I can see the following:

• Use something like simpleEscape as url.PathEscape won't work with ' (it should be replaces to '' to work)
• Use .GetFileByPath (if your SharePoint is 2016 or greater) instead of .GetFile to support filenames with %

func main() {
	// ...

	sp := api.NewSP(client)

	files, _ := sp.Web().GetFolder("/sites/ci/Shared Documents").Files().Select("ServerRelativeUrl").Get()
	fmt.Printf("Files count: %d\n", len(files.Data()))

	for _, file := range files.Data() {
		fmt.Printf("\nFile: %s\n", file.Data().ServerRelativeURL)
		serverRelativeURL := file.Data().ServerRelativeURL
		// serverRelativeURL = url.PathEscape(strings.Replace(serverRelativeURL, "/sites/ci", "", 1)) // won't work
		serverRelativeURL = simpleEscape(serverRelativeURL)
		// item, err := sp.Web().GetFile(serverRelativeURL).GetItem() // won't work with files containing % in the path
		item, err := sp.Web().GetFileByPath(serverRelativeURL).GetItem()
		if err != nil {
			fmt.Println(err)
			continue
		}
		data, _ := item.Select("Id").Get()
		fmt.Printf("Item ID: %d\n", data.Data().ID)
	}
}

func simpleEscape(s string) string {
	s = strings.Replace(s, "'", "''", -1)
	s = strings.Replace(s, "%", "%25", -1)
	return s
}

I can add simpleEscape to the library methods for files and folders getters.