Built-in server

[robertvazan]

I propose adding simple built-in Wikibase server to WDTK, mostly intended for use in unit tests. This issue is for discussion of the idea.

Built-in server would have some limitations, at least initially:

• Only in-memory storage. It might be exposed for direct access if it helps avoid complexity elsewhere.
• No human-readable error messages. Just a dump of whatever is found in the exception.
• Server would support only APIs currently used by WDTK client.
• Some hard to implement APIs might be mocked. Full-text search comes to mind.
• No optimizations. No security (except login). No monitoring/management. No extensions.

Such server would have a number of applications:

• Apps built on top of WDTK can use it in their unit tests as a fast, safe, and fully functional alternative to test.wikidata.org.
• WDTK client and server can be tested against each other.
• WDTK client tests can be cleaner, less fragile, and more comprehensive.
• Mockwebserver dependency can be dropped and okhttp downgraded. See Revert "Upgraded to JUnit5 (#536)" #694.

It looks like a lot of work, but given the above simplifications, I think it can be done in under 1K lines as long as the server systematically taps into existing functionality in XDocument and XUpdate classes. The investment will pay back during both WDTK and app development.

[thadguidry]

If it's mostly for unit tests, why not just run a Wikibase docker image as part of the CI/CD as well as for local development.
It seems like adding it into WDTK might not be that advantageous? Maybe the advantages could be added in your notes above.

[robertvazan]

Dockerized Wikibase server would be preferable, because it is complete and identical to production server. The trick is in how to fully automate test setup. Forcing every developer to manually setup Wikibase container and then configure tests to use it by placing config file in well-known location is rather unappealing.

[robertvazan]

PS: Wikibase API would also have to be reviewed to make sure it gives full control of the server. Tests need to create users, manipulate their permissions, and bulk-delete everything between tests.

[wetneb]

It seems like really a ton of work to me, and difficult to maintain. If we migrate to such a system, then it would also create the expectation that if someone implements support for a new action in WDTK, they should also implement it in this bare-bones server, and probably test that implementation there as well… it feels pretty daunting and overkill to me.

[robertvazan]

@wetneb I have done co-development of client+server+messages in my professional work. Improved testability of the full stack is totally worth the extra 10-20% of code per feature. Client has to be tested somehow. Thorough testing with mockwebserver is likely more expensive and less reliable. Dockerized Wikibase could be even better option though. I am reviewing that approach now.

[thadguidry]

@robertvazan let me know if you need help. or hop into the Telegram channel for Wikibase https://t.me/joinchat/HGjGexZ9NE7BwpXzMsoDLA