Code

new normalisation#510

Code

Closed

Dragos0000 wants to merge 134 commits intomainOP-TED/ted-rdf-conversion-pipeline:mainfrom feature/TED4-58OP-TED/ted-rdf-conversion-pipeline:feature/TED4-58Copy head branch name to clipboard

+2,958-2,497

Commits

Commits on Jul 4, 2023

• implement github download with credentials

  Show description for 7b109a6
  CaptainOfHacks

  committed
  7b109a6

• Fixed tests for notice_publisher feature

  kaleanych

  committed
  5461333

• Merge pull request #485 from OP-TED/feature/TED-1401

  Show description for 27b4228
  CaptainOfHacks

  authored
  27b4228

• Update conceptual_mapping_processor.py

  CaptainOfHacks

  committed
  aef716e

• Merge pull request #486 from OP-TED/feature/TED-1401

  Show description for c3a866a
  CaptainOfHacks

  authored
  c3a866a

• wip

  CaptainOfHacks

  committed
  0ff927d

• Merge pull request #487 from OP-TED/feature/TED-1401

  Show description for 61f8d9c
  CaptainOfHacks

  authored
  61f8d9c

Commits on Sep 8, 2023

• Update notice_processing_pipeline.py

  CaptainOfHacks

  committed
  49ac175

• Update DagBatchPipelineOperator.py

  CaptainOfHacks

  committed
  2503685

• Update notice_batch_processor_pipelines.py

  CaptainOfHacks

  committed
  9410465

• Update notice_processor_pipelines.py

  CaptainOfHacks

  committed
  619cc4b

• Update pipeline_protocols.py

  CaptainOfHacks

  committed
  7cb91d9

• Create MappingSuiteTransformationPool

  CaptainOfHacks

  committed
  52bc6e7

• Update documentation

  CaptainOfHacks

  committed
  188ef86

• Extend notice eligibility services

  CaptainOfHacks

  committed
  7f6d107

• Create notice_batch_transformer.py

  CaptainOfHacks

  committed
  6c68d78

• Update docker-compose-worker.yaml

  CaptainOfHacks

  committed
  56a8152

• Add AIRFLOW_NUMBER_OF_WORKERS env config

  CaptainOfHacks

  committed
  9cc2aab

Commits on Sep 12, 2023

• Update docker-compose.yaml

  CaptainOfHacks

  committed
  bcb162c

• Update docker-compose.yaml

  CaptainOfHacks

  committed
  c60ddf3

• Update Airflow version to 2.7.1

  CaptainOfHacks

  committed
  e41e97c

• Update rmlmapper to 6.2.1

  CaptainOfHacks

  committed
  9c3922a

• Update project requirements

  CaptainOfHacks

  committed
  f0d8ebd

• update sample application with new queries

  duprijil

  committed
  905c68b

• Create sample_application.xlsx

  duprijil

  committed
  05e7895

• Adjust tests and model logic to suport pydantic 2 version

  CaptainOfHacks

  committed
  4ac8b8b

• Merge pull request #489 from OP-TED/feature/TDA-272

  Show description for 899d41f
  CaptainOfHacks

  authored
  899d41f

• Update conftest.py

  CaptainOfHacks

  committed
  bacff55

• Update conftest.py

  CaptainOfHacks

  committed
  c3ff970

• Update entity_deduplication.py

  CaptainOfHacks

  committed
  0c44bfa

• Merge branch 'feature/TED6-37' into feature/TED6-23

  CaptainOfHacks

  committed
  850d48a

• Merge pull request #488 from OP-TED/feature/TED6-37

  Show description for 6adc67a
  CaptainOfHacks

  authored
  6adc67a

• Merge pull request #490 from OP-TED/feature/TED6-23

  Show description for c968cc7
  CaptainOfHacks

  authored
  c968cc7

• fix AirflowNumber of workers config

  CaptainOfHacks

  committed
  8a756b1

• add ignore warning deployment exposure

  CaptainOfHacks

  committed
  6f81d83

• Merge pull request #491 from OP-TED/feature/TED6-23

  Show description for 3274fd0
  CaptainOfHacks

  authored
  3274fd0

Commits on Sep 13, 2023

• Sample application page moved to another repo

  duprijil

  committed
  2e84ddb

• delete unnecessary images

  duprijil

  committed
  616d67b

Commits on Sep 15, 2023

• Create test_new_trigger_ui.py

  CaptainOfHacks

  committed
  f762e74

• delete unnecessary files

  duprijil

  committed
  d51dbb3

Commits on Sep 17, 2023

• Merge pull request #492 from OP-TED/feature/TED-1416

  Show description for 47c8a01
  costezki

  authored
  47c8a01

Commits on Sep 18, 2023

• Create test_dag_a.py

  CaptainOfHacks

  committed
  5962693

• Update test_dag_a.py

  CaptainOfHacks

  committed
  6a288cd

• Update test_dag_a.py

  CaptainOfHacks

  committed
  aaec47c

• Update test_dag_a.py

  CaptainOfHacks

  committed
  78f82f0

• Update test_dag_a.py

  CaptainOfHacks

  committed
  e6e035e

• Create test_dag_b_trigger.py

  CaptainOfHacks

  committed
  8540649

• implement DAGs trigger UI

  CaptainOfHacks

  committed
  61120b5

• Update fetch_notices_by_date.py

  CaptainOfHacks

  committed
  6d0b7b7

• Update fetch_notices_by_query.py

  CaptainOfHacks

  committed
  782fa58

• Update fetch_notices_by_date.py

  CaptainOfHacks

  committed
  f01dd75

• Update fetch_notices_by_query.py

  CaptainOfHacks

  committed
  4fc420d

• remove test dags

  CaptainOfHacks

  committed
  5da8d5e

• Revert "remove test dags"

  Show description for f187842
  CaptainOfHacks

  committed
  f187842

• add DAGs documentation feature

  CaptainOfHacks

  committed
  7f48819

Commits on Sep 19, 2023

• Merge pull request #493 from OP-TED/feature/TED6-33

  Show description for 0b8c19a
  CaptainOfHacks

  authored
  0b8c19a

• minimize project requirements

  CaptainOfHacks

  committed
  9840689

• Update requirements.txt

  CaptainOfHacks

  committed
  2a19e37

Commits on Sep 20, 2023

• Merge pull request #494 from OP-TED/feature/TED6-40

  Show description for 6eeb9b3
  CaptainOfHacks

  authored
  6eeb9b3

• Update DagBatchPipelineOperator.py

  CaptainOfHacks

  committed
  8534c81

• Update DagBatchPipelineOperator.py

  CaptainOfHacks

  committed
  acf52b4

• Update DagBatchPipelineOperator.py

  CaptainOfHacks

  committed
  0764261

• Merge pull request #495 from OP-TED/feature/TED6-25

  Show description for 646b688
  CaptainOfHacks

  authored
  646b688

Commits on Sep 23, 2023

• Update fetch_notices_by_date.py

  CaptainOfHacks

  committed
  6a62161

• Update fetch_notices_by_date_range.py

  CaptainOfHacks

  committed
  c55262d

• Update DagBatchPipelineOperator.py

  CaptainOfHacks

  committed
  5a5807b

• fix processed notice ids

  CaptainOfHacks

  committed
  0937151

• Merge pull request #496 from OP-TED/feature/TED6-50

  Show description for 946a9d2
  CaptainOfHacks

  authored
  946a9d2

Commits on Sep 25, 2023

• Update event_message.py

  CaptainOfHacks

  committed
  9c4bdda

• Update event_message.py

  CaptainOfHacks

  committed
  2c26e0d

• Update DagBatchPipelineOperator.py

  CaptainOfHacks

  committed
  e0f85c7

• wip

  CaptainOfHacks

  committed
  b6b10b6

• wip

  CaptainOfHacks

  committed
  e7a3c64

• WIP

  CaptainOfHacks

  committed
  996306a

• Update notice_processor_pipelines.py

  CaptainOfHacks

  committed
  fd505a4

• Update DagBatchPipelineOperator.py

  CaptainOfHacks

  committed
  f4f8487

• Merge pull request #497 from OP-TED/feature/TED4-55

  Show description for 4905038
  CaptainOfHacks

  authored
  4905038

Commits on Sep 27, 2023

• Fix when fetch already fetched notices don't remove old lazy_fields

  CaptainOfHacks

  committed
  5416dd7

• Merge pull request #498 from OP-TED/feature/TED4-57

  Show description for e491e5d
  CaptainOfHacks

  authored
  e491e5d

Commits on Sep 29, 2023

• implement multiple publish channels

  duprijil

  committed
  8449c26

• change ENABLED config var with BUCKET and HOST

  duprijil

  committed
  ef84a41

• Merge pull request #499 from OP-TED/feature/TED6-32

  Show description for d39aff1
  duprijil

  authored
  d39aff1

• metabase version update

  Dragos0000

  committed
  54084cc

• make cellar url configurable from env and airflow

  duprijil

  committed
  e9b5804

• fix incorrect type for string values

  CaptainOfHacks

  committed
  b59a77a

• rename to CELLAR_WEBAPI_SPARQL_URL

  duprijil

  committed
  7b5d632

• Merge pull request #502 from OP-TED/feature/TED4-62

  Show description for 88b6af3
  CaptainOfHacks

  authored
  88b6af3

• Merge pull request #501 from OP-TED/feature/TED4-21

  Show description for 4870847
  duprijil

  authored
  4870847

Commits on Oct 2, 2023

• create DailyNoticesMetadata model, repository + tests

  duprijil

  committed
  4296f0e

• WIP

  duprijil

  committed
  f5fba8e

Commits on Oct 3, 2023

• WIP

  duprijil

  committed
  56cbb7f

• WIP

  duprijil

  committed
  b7d4389

• add cumulative frequency

  CaptainOfHacks

  committed
  6c4f614

• WIP

  duprijil

  committed
  262d3fb

• WIP

  duprijil

  committed
  272d0a8

• WIP

  duprijil

  committed
  78440f7

Commits on Oct 4, 2023

• prepare tests

  duprijil

  committed
  102ebf1

• Update conftest.py

  duprijil

  committed
  e61ab85

• Update yarrrml2rml_converter.py

  duprijil

  committed
  107eb9c

• update depricated pydantic functions

  duprijil

  committed
  a1eb832

• Merge pull request #504 from OP-TED/feature/TED4-65

  Show description for 9033665
  duprijil

  authored
  9033665

• Merge branch 'main' into feature/TED4-10

  duprijil

  committed
  92044d4

• Merge branch 'main' into feature/TED4-49

  duprijil

  committed
  7298fff

• make changes according review

  duprijil

  committed
  a45c5d3

• Merge pull request #505 from OP-TED/feature/TED4-49

  Show description for 6c12c06
  duprijil

  authored
  6c12c06

• Merge branch 'main' into feature/TED4-10

  duprijil

  committed
  97b6258

• update conftests

  duprijil

  committed
  4f63665

• Update test_notices_metadata_services.py

  duprijil

  committed
  7440de7

• Merge pull request #500 from OP-TED/feature/TED4-29

  Show description for 6eb5687
  Dragos0000

  authored
  6eb5687

• Merge pull request #503 from OP-TED/feature/TED4-10

  Show description for 095801d
  duprijil

  authored
  095801d

Commits on Oct 6, 2023

• resolve bug

  duprijil

  committed
  70be603

• Update daily_notices_metadata_services.py

  duprijil

  committed
  6ee10a5

• Update test_notices_metadata_services.py

  duprijil

  committed
  ba3cf16

• Update daily_notices_metadata_update.py

  duprijil

  committed
  3e3fcbf

• Update daily_notices_metadata_update.py

  duprijil

  committed
  dd053ed

• Update daily_notices_metadata_update.py

  duprijil

  committed
  da1977c

• make start picking at dag level

  duprijil

  committed
  b67146b

• Update daily_notices_metadata_update.py

  duprijil

  committed
  2733359

• Update daily_notices_metadata_update.py

  duprijil

  committed
  b1c5a1c

• Update daily_notices_metadata_update.py

  duprijil

  committed
  a31ae28

• Merge pull request #506 from OP-TED/feature/TED4-66

  Show description for e26fe8b
  duprijil

  authored
  e26fe8b

Commits on Oct 10, 2023

• code cleaning and update to pydantic v2

  CaptainOfHacks

  committed
  774c953

Commits on Oct 11, 2023

• Merge pull request #507 from OP-TED/feature/TED4-64

  Show description for 39e3df4
  CaptainOfHacks

  authored
  39e3df4

• optimise e2e tests

  CaptainOfHacks

  committed
  318697e

• clean tests

  CaptainOfHacks

  committed
  231af55

• Merge pull request #508 from OP-TED/feature/TED4-67

  Show description for 8c86cd0
  CaptainOfHacks

  authored
  8c86cd0

• add filter by mapping_suite_id for re-transform DAG

  CaptainOfHacks

  committed
  14b0319

• Merge branch 'main' into feature/TED4-22

  CaptainOfHacks

  committed
  0703cb1

• Merge pull request #509 from OP-TED/feature/TED4-22

  Show description for 5aa4c19
  CaptainOfHacks

  authored
  5aa4c19

Commits on Oct 13, 2023

• extreactor adapters

  Dragos0000

  committed
  4bc2a87

• define new notice extractor and normaliser architecture

  CaptainOfHacks

  committed
  c4c23b3

Commits on Oct 20, 2023

• first draft WIP

  Dragos0000

  committed
  2005615

• Update test_notice_packager.py

  CaptainOfHacks

  committed
  6e5260f

Commits on Oct 23, 2023

• test passing

  Dragos0000

  committed
  80b60a7