Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HIGHLIGHT engine and major restructure #152

Merged
merged 59 commits into from
Jun 14, 2024

Conversation

ator-dev
Copy link
Member

@ator-dev ator-dev commented Feb 5, 2024

The Custom Highlight API, now available in Chromium-based browsers and under development in Firefox (bugzilla) has made it possible to highlight webpages without changing their structure (or even, their attributes).

This new first-class support for the highlighting paradigm warrants a new highlighting engine to take full advantage of it in supported browsers. It creates potential for any modern "search highlighter" browser extension to operate with high efficiency, high accuracy, and low-to-none website experience impact - all of this with an API which is much easier for developers to work with, enabling more maintainability and less technical debt.

I have taken the opportunity to fully restructure my codebase! The new Mark My Search, making use of small, independent ES modules and classes, will be highly maintainable and should be easier to learn from.

I am aiming for full feature parity with my other highlighting engines before merging this PR. My other aims before the release of v1.15 v2 include unit tests for the highlighting engines, the ability to choose any highlighting engine, and improvements to how messages are handled internally.

Fix #144

- Add beginnings of HIGHLIGHT engine
- Prepare for re-use of PAINT highlighter components within HIGHLIGHT
- Add initial HIGHLIGHT implementation which can only do [responsive]
  highlighting; currently does not clean up after nodes are deleted
- Fix mistakes in last commit
- Temporarily enable HIGHLIGHT all the time
- Make HIGHLIGHT able to (supposedly) highlight the whole document in
  ranges correctly - and maintain correct state responsively - as well
  as PAINT
- Make "Special" highlighting engine able to highlight inside inputs
  - Currently the highlight boxes are not placed correctly
- Many small improvements to highlighting code
- Add a modules/ directory in src/ to hold re-usable ES modules
- Create a common module with functions and types used across the whole
  content script
- Create a highlight/ directory in src/modules/ to hold all
  highlighting-related modules
- Split off all highlighting components into modules importing from each
  other
- Do some of the work towards making imports work in the browser
  extension environment
  - Add a script to automatically convert TypeScript-friendly imports
    into browser-friendly imports
- Edit manifest to expose module scripts
- Fix (and more exactly specify) many top-level imports
- Split Highlightability (Abstract and Standard implementation) into
  separate module from flow-monitor
- Move CSSPaintHighlightability into CSS Paint method file
- Convert scripts previously in /include into modules, pulling out
  several parts to make smaller, more specific files
- Make use of static imports (over dynamic import()s) everywhere EXCEPT
  where, in content-only modules, it makes sense to conditionally import
  code
- Convert content.ts into a module (content.mts) in order to make use of
  static imports; run content-entry.ts, which dynamically imports
  content.mts in one line
- Convert enums (impossible to transfer neatly between modules, and
  generally problematic) into objects and union types (depending on the
  case)
- Overall make the extension functional again, save for likely bugs and
  a few temporarily disabled features
- Modularise some of the background script
- Modularise more of the content script, including the toolbar and
  stylesheet parts
  - Currently includes legacy hack for setting terms
- Make pages work again using modules
  - Highlighting in internal page not yet possible (if content script is
    injected and used, highlighting will not activate in any regular
    tabs)
- Make term occurrence checks work again, this time by putting the
  highlighting engine in charge of creating and invoking
  indicator/term-controls refresh generators
- Improve organisation of some code, mainly types
- Separate message module into type module, background-messaging, and
  tab-messaging
- Rename "highlight messages" to "tab messages"
- Move modules requiring background privilege into modules/privilege/
- getRequest{WaitDuration,RequestReschedulingDelayMax}() class functions
  were remnants of the old term-occurrence-refresh architecture
- Create concept of highlighting _models_, which organise modules used
  in highlighting engines by the model of storing and transforming
  highlights
- Each highlighting engine conforms to exactly one of the models
  - ELEMENT: TreeEdit (manipulates the DOM tree to store highlights
    directly)
  - HIGHLIGHT, PAINT: TreeCache (links highlighting information to the
    tree without changing its structure)
- Organise highlighting modules by model as appropriate
- Separate ELEMENT engine tags into tags module in TreeEdit model
- Split term-counting, term scroll markers (term-marking), and
  jumping/stepping (term-walking) from PAINT (and also ELEMENT) into
  modules, distinguished and grouped by the new 'highlighting models'
  concept
- Simplify the AbstractEngine interface by incorporating the new classes
  introduced with these modules, via composition
- Hence simplify the PAINT and ELEMENT main modules
- Incorporate the new classes into HIGHLIGHT, causing nearly all
  features to be activated 🎉
  - Scroll markers are not quite activated, as they require an array of
    all actively highlighted elements, which is currently not readily
    accessible from HIGHLIGHT
- (HIGHLIGHT) delete highlights when they are condemned by FlowMonitor
- (HIGHLIGHT) match request parameters for refreshing scroll markers
  (not implemented) to PAINT
- Reorder FlowMonitor's constructor parameters
- Fix unnecessary deletion of entire highlights in last commit by
  creating proper abstractions over highlights and highlight registries,
  the former of which automatically maps highlights by more useful keys
- Experimentally give highlights an opacity of 0.6
- Implement scroll markers for HIGHLIGHT
- Fix scroll markers algorithm for PAINT and HIGHLIGHT
- Change delivery of BoxInfo population/removal information by
  FlowMonitor
- Do not add TreeCache to entire tree in HIGHLIGHT, strive to remove
  when no longer needed
  - Remove incorrect assumptions about TreeCache existing
- Add better typing for TreeCache (Flow type info currently inconsistent
  between algorithms, also some mess)
- **I finally noticed that I was still using /dist/background.js**,
  which only worked because it existed in my compilation cache; this has
  been changed to the correct path, /dist/background.mjs
- Fix my .eslintignore rule for modules (it should be .mjs, not .jsm)
- Remove Dummy versions of classes, instead just assigning `undefined`
  to certain properties which will later be assigned an instance of an
  implementation of an interface
- Finally bump minor version to 15
- Make flow-monitor and engine modules contain only type definitions
  (abstract interfaces, specifically)
- Make independent modules e.g. for TreeCache and CACHE
- Rename highlighting to highlight-tags
- Split functions from engine module into container-blocks and
  page-updates, making engine a type-only module
- Add stopgap fix for choosing PAINT method in background
- Do not add highlighting cache to every element, make PAINT and flow
  monitor code work correctly when highlighting cache cannot be assumed
- Fix PAINT method assignment
- Improve code style
- Cut down MatchTerm to its essential, deterministic parts (phrase and
  matchMode)
- Pass hues wherever hues are needed (as opposed to sometimes accessing
  one from a MatchTerm)
- Create a registry of term tokens and a registry of term patterns, pass
  them around where needed (similarly to hues)
- No longer rely on a given term knowing its keyboard commands
- Make many class attributes readonly when they only need to be changed
  in the constructor
- Make some "optional" class attributes guaranteed, modify accordingly
- Merge changes which replace the original basic [internal] storage API
  with a much safer, more capable one
- Improve the API by namespacing (creating an abstract Bank class for
  Bank storage functions, an abstract Config class for Config storage
  functions)
- Merge other changes to the background script from the storage branch
@ator-dev ator-dev force-pushed the highlight-api-engine branch from cfb2ca7 to a4889b6 Compare May 28, 2024 23:36
- Note: Currently largely non-functional
- Refactor entire toolbar module to use an object-oriented style with
  interfaces and classes
@ator-dev ator-dev force-pushed the highlight-api-engine branch from a4889b6 to 57afacc Compare May 28, 2024 23:52
ator-dev added 2 commits May 29, 2024 02:21
- Note: Still significantly dysfunctional
- Break apart monolithic "toolbar" module into smaller modules
  responsible for specific functions
- Make most aspects of the toolbar fully functional again
- Do not create toolbar before controls info has been (is likely to have
  been) populated
- Group term setting functions under object implementing TermSetter
  interface; pass it under more restricted interfaces to term
  replace/append controls
@ator-dev ator-dev force-pushed the highlight-api-engine branch from 0d8f49d to 6807125 Compare June 7, 2024 23:48
ator-dev added 17 commits June 8, 2024 01:23
- Make menu opening work as intended [bugs may still be present]
- Preserve the menu opener (currently, only term inputs) in the right
  cases
- Always reset input value when committing, hence preventing the entered
  value from being left in the term append control.
- General clean ups
- Take adavantage of certain highlighting engine objects/methods being
  non-nullable now
- Remove "Standard" from the names of each single implementor of an
  interface, and rename their files to just be the class name (in
  hyphen-snake-case form)
- Allow holding Shift when pressing a match-mode letter to toggle the
  option without closing the menu
- Ignore letter presses on the match-mode menu which do not correspond
  to an option
- Pass all events with Ctrl, Meta, or Alt through the whole toolbar, to
  avoid interfering with browser-provided keyboard controls
Changes intended to bring toolbar to feature parity (partial list):

- Improve intra-term shifting using arrow keys
- Do not reset term input value to old value when committing
- Rewrite toolbar focus (etc) state handling for consistency and
  flexibility
  - Correctly return focus to term input after dragging down menu and
    when closing, and when using other controls/components
  - Prevent returning focus to term input immediately after pressing on
    the control content triggered "jumping" to the term
- Add `toggleBarHidden()` method to Toolbar and AbstractToolbar
- Call from content instead of adding class manually
- Exclude other interfaces from AbstractToolbar
- Refactor toolbar interface a little
- Use toolbar interface instead of querying for bar directly in content
  script main
- Ensure that its visibility is kept up to date
- Dragging of toolbar icons was unintentional (draggable="false" for
  some reason doesn't work), and only prevented being able to
  consistently use the 'pull down menu' convenience
- Fix the age-old issues with term input selection consistency once and
  for all (OMG)
  - Ensure term input is always kept up to date with its corresponding
    term
- Refactor option list's "make element full/pulldown trigger" methods,
  describe the situations in which certain event functions are triggered
- Ensure that "just released on element" state is always reset in
  `makeElementPulldownTrigger()` method's window pointerup listener
- Allow option list pulldown from element based on whether corresponding
  class [new] is present
- Only add "pointerup" event listeners when waiting for the event after
  "pointerdown"
@ator-dev ator-dev force-pushed the highlight-api-engine branch from 85ea76b to 0f9092d Compare June 12, 2024 16:23
ator-dev added 3 commits June 12, 2024 19:05
- Small changes to make code more readable
- Fix error in Toolbar's `indicateTerm()` if no term was previously
  indicated
- Rename content's `respondToCommand()`, use a function to produce it
  instead of a generator function
- Refactor some of `respondToCommand()`
- Use a new method of highlight style differentiation when terms have
  the same colour and HIGHLIGHT engine is in use
  - A very limited set of CSS properties can be used in ::highlight();
    this includes `text-decoration`
  - Cycling through `text-decoration-style` of dashed, dotted, and solid
  - Every `text-decoration-style` cycle, use a larger
    `text-decoration-thickness` and less highlight background opacity;
    currently only 2px or 3px thickness
  - `text-decoration-color` of highlight colour (but much more visible
    than background, and fully opaque)
@ator-dev
Copy link
Member Author

This PR has turned into a monster that changes far more than any single PR should. The largest and most obvious change - that is, the restructuring and modularisation of nearly the entire codebase - isn't even much related to the original intent; although modularisation certainly made the development of the HIGHLIGHT engine more interesting. Moving forwards, Mark My Search's codebase should be far more flexible and fun to work with.

The latest commit on this branch brings HIGHLIGHT to feature parity with the other engines. It should require relatively little work to make it release-worthy. Hence I am merging this PR here.

@ator-dev ator-dev merged commit 395ed93 into searchmarkers:main Jun 14, 2024
@ator-dev ator-dev deleted the highlight-api-engine branch June 14, 2024 13:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

New highlight method
1 participant