-
Notifications
You must be signed in to change notification settings - Fork 0
Work Plan
Earlier ideas for this project fell into two categories of interest: (1) a gamified experience of the Library's Free to Use and Reuse Collection that targeted younger audiences and (2) a collage tool for creatives that would allow for artistic exploration of the Library's Free to Use and Reuse Collection.
This project will combine both of these ideas by having the user create a collage out of selected images as a way to "explore" the history of certain locations and historical figures. The experience will look like:
User enters the collage interface. User selects a city "Main Street" (for example: San Fransisco, 1920s, from the Main Street collection within the Free to Use and Reuse photo collection). A short blurb with a historical fact appears, and the selected image becomes the backdrop for the user's collage.
Now, the user has a sidebar available to them with Library collection photographs from their chosen city, of historical significance (for example: historical figures, artifacts, flora and fauna, etc). When the user clicks on an image to use in their collage, the image will first show a text pop-up with a little blurb about the photo's history and connection to the city, and then the image will "segment" into different pieces that the user can move around, rotate, and resize in the collage. For example, a photo of a historical figure might segment into the person, the person's hat, and the person's briefcase.
The goal is to have students learn about a city's lesser told history. The collage aspect provides an end goal that entices participation, while the gamified aspect ensures interaction with the material in a meaningful way.
The project will be broken into three main phases.
Phase 1: Selecting the cities and choosing photos -- the research phase.
Phase 2: There are two somewhat simultaneous working parts here. The image segmentation and object recognition aspect of the deliverable will involve experimentation with machine learning, while the interface will require different tools to build.
Phase 3: Testing.
Age group: Later elementary students (4th and 5th grade). There is some "leakage" here into early middle school.
Use case: Civics/history. We envision that this can be part of a specific civiv/history lesson if a teacher wishes. The more likely scenario is an "educational play." (In elementary school, if you finished your work early or are in after school care, we'd be able to play educational games on the computers.)
Note: We are still exploring the exact set of images we'll use from the Free to Use and Reuse Collection.
Image Instance Segmentation
- labelme, Image Annotator tool: In many ways, I envision the collage tool working like the AI powered annotation features of this tool; enabling individuals to select and participate in the masking process before an item in the collage is cut-out: GNU General Public License
- EfficientSAM: Leveraged Masked Image Pretraining for Efficient Segment Anything: This is the AI Model used in the labelme Annotator tool (labeled: EfficientSAM (Speed) that Aisaiah sampled with the NY content from the Free-to-Use. It has been reliable and quick so far in annotating; Apache-2.0 license
- Mask_RCNN.ipynb: A notebook from a tutorial on background remova. This is an example of image masking and background removal.
- COCO Dataset: This is a small Computer Vision training dataset containing 80 categories of common objects in context. Should we elect to train a Computer Vision model this could be combined with formatted dataset as a basis.Creative Commons Attribution 4.0 License
Interactive Game
Generally speaking these are some of game developer tools.
- [Unity] (https://unity.com/)
- Scratch
- Interface hosted by the LoC or github pages that gives individuals access to an Library Collage Tool and an Interactive Game that leverages attributes of the tool.
List the major milestones, and a brief description and due date for each. Note that major deliverables should be complete by around week 8 (~July 12) to ensure enough time for Display Day prep and offboarding activities