Skip to content

Commit

Permalink
Release v0.14.0 (#1078)
Browse files Browse the repository at this point in the history
* feat(otel): create a config to decide which way to scrape metrics

* fix(orb-ui): fix not displaying agent matches on creation

* feat(otel): code review changes

* fix(orb-ui): fix menu items order in navigation menu

* feat(otel): code review changes

* fix(orb-ui): fix agent view copy command button not changing icon after click

* test(otel): create unit test to otlp exporter

* test(otel): using otlp exporter on orb agent

* fix(orb-ui): #625 fix order on dashboard page (#637)

* fix(orb-ui): fix order of orb entities on dashboard page

* fix(orb-ui): fix wrong row ordering

* english correction

* feat(otel): wip code sanitization

* feat(otel): wip extract code to new methods

* feat(otel): Create otlp exporter starter on pktvisor startup.

* enhancement(orb-ui): improve home page card ordering for user actions

* fix(dataset): Fix http patch to put on edition.

* feat(otel): Make otel configuration parameters more readable when configuring pktvisor

* bump to 0.11.0-develop

* feat(otel-exporter): Fix merge conflicts with ui

* fix ui conflicts

* fix missing PKTVISOR_TAG in GitHub actions. bring go builds up to 1.17 (#664)

* fix missing PKTVISOR_TAG in GitHub actions
* bring go builds up to 1.17
* add agent README

* test(sinks): increase unit test coverage at data access layer

* test(sinks): increase service unit test coverage

* test(sinks): increase endpoint unit tests coverage

* add dnstap input (#669)

* fix(orb-ui): fix button outside form scope not submitting forgotPassword

fix(orb-ui): override nbResetPassword resetPassword method

* feat(otel-exporter): create otel otlp exporter starter

* test(sinks): increase endpoint unit test coverage

* bug(sinker): Applying changes from pktvisor/types.go to sinker/pktvisor/types.go to fix prom particle generation

* test(sinks): fix test cases description

* fix(orb-ui): remove option to choose backend type for agent policy

* fix(orb-ui): tap & input of agent policy

fix: use input predefined in tap configuration

fix: show input in dropdown selector for tap

fix: fail gracefully - add label stating there are no taps listed for when
there are no agents running. Links to Agents List Page.

* fix(orb-ui): fix handler configuration json not being posted

* bug(sinker): wip sinker console output

* bug(sinker): wip remove unused console output

* bug(sinker): wip ugly way to solve the topGeoLoc and topASN problem

* testing agents through the ui (#667)

* testing agents through the ui

* bug(sinker): wip solving TopGeoLoc using regex

* test(sinks): fix postgres unit test description and fix error return for update non-existing sink

* test(sinks): fix expected error of postgres unit test cases

* add the indirect go mods, ensure static link during CGO build.

* bug(sinker): wip TopGeoLoc

* bug(sinker): wip using regex to get exceptions to prometheus conversion

* bug(sinker): return version to unknown on version.go file

* bug(sinker): fixing go.mod file

* fix(orb-ui): Page not responding after clicking on Register nor logging into dashboard

* feat(sinker): remove unnecessary comments and console writers

* feat(orb-ui): add advanced options to tap configuration and filter configuration in agent policies.

* feat(fleet): remove last_db_data and agent_metadata from agent list endpoint

* feat(orb-ui): mark text where user could/should change interface

* feat(orb-ui):

* remove odd <hr/> breaks
* add labels to each section of input/tap configuration

* feat(orb-ui): change color for marked text

* remove codeql analysis for now

* feat(orb-agent): extract pktvisor otel receiver / exporter functions to a new file

* feat(orb-agent): Remove policy from agent local repo on removal action

* 679 second policy (#683)


* apply two policies to an agent

Co-authored-by: Stéfano de Faria <[email protected]>

* fix ui conflicts

* fix conflicts with python-test

* fix(orb-ui): chain observables for dataset related entities and display name of such entities when retrieved

* fix(orb-ui): Fix wrong spacing between sink display and elements above it

* fix(orb-ui): keep sinks list instead of only their names

* refactor(orb-ui): cleanup unused imports and code style

* refactor(orb-ui): implement ngOnDestroy to unsubscribe from observables

* fix(orb-ui): remove option to filter dataset by tags

* recovering some changes that I noticed that shold be on previous PR (#724)

* refactor(orb-ui): refactor pactsafe integration

* refactor(orb-ui): finish integration with pactsafe

* show only checkbox with 'agreement' label
* remove minified pactsafe wrapper
* contract is now shown when user clicks link

* int_tests - automate account registration with credentials required (email and password) if they are not yet registered (#726)

* register an account using email and password if credentials are not registered yet

* fix(orb-ui): Policy Details View (#734)

* wip

* fix(orb-ui): Add more details on Agent Policy Details View

* add handlers section to display name of each handler
* add input type and tap name section

* feat(orb-ui): list handlers as accordion items

* show handler details inside accordion body, represented in JSON

* Feat/orb agent debug arg (#728)

* feat(orb-agent): Create debug parameters to agent

* feat(orb-agent): Use right debug configuration args

* fix(orb-ui): #737 register button not working (#740)

* fix(orb-ui): register button not working

* fix(orb-ui): enable sandbox when not production build

* fix(orb-ui): register button not working - _ps module not available

* Bug/mqtt incoming stoped error (#742)

* feat(orb-ui): add advanced options to tap configuration and filter configuration in agent policies.

* feat(orb-ui):

* remove odd <hr/> breaks
* add labels to each section of input/tap configuration

* recovering some changes that I noticed that shold be on previous PR (#724)

* fix - upgrade paho.mqtt version and setup auto reconnect configuration

* remove ui e python-test

* merge fix with ui and python-test

* bug(orb-agent) Change keep alive and ping timeout configuration for bigger times

Co-authored-by: gpazuch <[email protected]>
Co-authored-by: manrodrigues <[email protected]>

* fix(orb-ui): disable ps contract box if not prod (#741)

* fix(orb-ui): register button not working

* fix(orb-ui): enable sandbox when not production build

* fix(orb-ui): register button not working - _ps module not available

* fix(orb-ui): hide contract box when development and not production

* fix(orb-ui): remove unused imports (#744)

* feat(orb-ui): Display warning when no tags added for agent (#735)

* Show message in agent add and agent list page warning user that agent has no tags to it.

* Also fix string for tag color on sink page

* Automated tests for delete a policy applied to an agent (#743)

Automated tests for delete a policy applied to an agent

* fix(orb-ui): Add missing required asterisk (*) on handler label field (#753)

* doc(policy): update dataset and agent policy creation request (#635)

* Feat/orb policy associate schema version (#732)

* feat(orb-agent): Create schema version field on policy and receive on policy creation

* feat(orb-agent): Update openapi spec to reflect schema version new field

* feat(fleet): Create schema version on fleet backends return

* feat(policy): Changes and enhancements after code review

* Bug/orb sinker ipv6 metrics (#755)

* bug(sinker): change colon for semicolon when creating prom labels (avoid problem of ipv6 hex names, which has colons)

* bug(sinker) change split method on label creation

* bug(sinker): improve log messages when prometheus components fail to be created

* bug(sinker): change log message type to error

* bug(sinker): change colon for semicolon on quatile lable

* delete dataset tests (#759)

* delete dataset tests

* Tests/login (#761)


* automated tests for login

* fix(orb-ui): add 'e.g.: ' to examples (#757)

* fix(orb-ui): fix error message during register (#758)

* authentication module no longer expects valid token on /users endpoint

* remove code for redirection on /users endpoint response - force auth

* fix(orb-ui): fix agent policy not able to apply when filter is empty (#762)

* fix(orb-ui): expose pactsafe env vars to docker space in local builds (#770)

* fix(orb-ui): expose pactsafe env vars to docker space in local builds

* building orb-ui via `make ui` -> make sure PS_SID and PS_GROUP_KEY are present in the environment.

* Fix whitespace

* fix(orb-ui): Fix environment setup

When reading from environment, setenv.js would check if property is available, and determine
wheter to produce the output to the environment file supplied to Angular App.

Since Dockerfile was patched to pass down such variables, they're now always present in the docker env.,
though set as empty string ''.

Setenv.js check changed to string comparisson.

* fix(orb-ui): check only that PS can be enabled, no longer check for prod

* refactor(orb-agent): enhancement on log messages (#750)

* refactor(orb-agent): enhancement on log messages
Create more wrappers for errors messages related to mqtt errors

* feat(orb-agent): remove unnecessary param version on agent creation

* Enhancement on logs for orb-agent policy removal

* security(org-agent): remove from logs key which is used to log on mqtt

* Fix policies migrations error (#774)

* Fix migrations error setting valid value for schema_version column creation

* bug(policies) make dafault value for schema_version 1.0

* Bug/orb mqtt agent group order creation (#772)

* wip investigation of the problem

* wip to solve bug on agent group order creation

* Wip create connection with agent group on agent creation

* Remove docker-compose auth changes which is unrelated this feature

* Fix unit test on agent view (missing mock implemetation for RetrieveAllByAgent method)

* bug(fleet): logs improvements and removal of unnecessary commets

* Change agent_group_id to group_id to keep consistency

* Handle error for retrieve agent group when connect agents to group channel

* New version 0.12.0-develop (#777)

* changing is_credentials_registered variable after register the account (#786)

* get correct agent through UI (#787)

* Remove orb_tag validation on agent edition (#789)

* change title of prometheus configuration (#790)

* Feat/orb agent handle multiple datasets (#781)

* feat(orb-agent): make orb-agent handle with multiple datasets for the same policy (avoid apply policy multiple times)

* initial logic for updating a policy

* Create logic to handle policy updated based on version

* create a timeout parameter on pktvisor request method

* Enhancements after code review

* removing duplicated tests (#794)

* automated tests: covering integration scenarios (#799)

* automated tests: covering integration scenarios

* Feature/orb policies increase unit tests coverage (#782)

* test(policies): improvements on unit tests coverage of data acess layer

* test(policies): improvements on service unit test coverage

* test(policies): improvements on unit test coverage of policies API

* Orb - tests documentation (#785)

* orb_int_tests_description

* Feat/orb policy last modify (#800)

* Fix schema_version empty valeu

* create last_modified field to registry the date of policy edition

* create last modified field on endpoint response

* Refactor/orb sinker change top k metrics label (#795)

* wip refactor prom particle label for top k metrics

* minor corrections

* create sinker parser for top n metrics

* remove unnecessary console prints

* changes requested on code review

* Bootstraping unit tests to sinker process metrics | mapped dhcp metrics

* Enhancements on unit tests

* remove migrations from this PR

* Tests/integration tests (#803)

* automated tests: covering integration scenarios

* fix(orb-ui): fix advancedOptions filter (#783)

* fix filter advancedOptions, enabling correct rendering of advanced and non-advanced options for agent policies

* Feat/orb agent reset (#816)

* wip orb-agent reset logic

* wip create endpoint to receive orb-agent reset request

* create logic for backend reset
enhancements on logs for backend reset procedure
create comms for receive agent reset message

* rollback build info version to unknown

* changes after code review

* build debug agent in ci

* Avoiding NaN on prometheus sending metrics every 1 minute even when its 0 (#818)

* set pktvisor port test (#823)

* (hotfix): node/types specify version 12.20.42 - 12.20.43 breaks other deps (#824)

* Fix dataset overriting to empty when edit a policy (#826)

* Feat/orb policy last modify (#819)

* Fix schema_version empty valeu

* create last_modified field to registry the date of policy edition

* create last modified field on endpoint response

* Add last modified field for list policies endpoint

* fix typo on agent tags help message (#834)

* add undersocre bofore ASN (#835)

* Bug/orb agent tag edition (#828)

* Fix dataset overriting to empty when edit a policy

* wip solve orb-agent tag edition problem

* enhancements on logs for group unsubscrition

* wip tags edition

* code sanitization

* fix group info struct name

* changes after code review

* fix(orb-ui): (#837)

* require two names, space separated, for full user name
* send full user name as signer id when contract agreement
* pactsafe confirmation email set to true - user should receive a copy of contract

* fix(orb-ui): (#838)

* agent provisioning command to use 'develop' tagged docker image -> ns1labs/orb-agent:develop

* creating unit test and fixing topASN label (#847)

* fix(orb-ui): agent policy fixes (#843)

* feat(orb-ui): agent policy handler add modal

* wip - agent policy adjustments

* fix(orb-ui): qname validator and other fixes

* fix(orb-ui): #487 last modified field on policies list page (#844)

* fix last modified date on agent policy list

* agent subscription to multiple groups (orb-tags) and  tag scenarios

* agent subscription to multiple groups (orb-tags) and  tag scenarios

* fix(orb-ui): fix regex pattern for fullname in register form (#845)

* Bug/orb mix tags (#848)

* trying approach with group tags

* merge tag columns to fix bug on matching tags

* fix(orb-ui): fix agent policy edit failing (#855)

* fix(orb-ui): Add cancel button to all stepper controls in forms (#846)

* fix(orb-ui): Add cancel button to all stepper controls in forms

* Agent Create/Edit
* Agent Group/Edit
* Agent Policy Create/Edit
* Dataset Create/Edit
* Sink Create/Edit

* fix(orb-ui): fix strings standardization and codestyle

* add separate orb-agent workflow

* add push step to new orb-agent workflow

* fix(orb-ui): fix validator for having at least one module in agent policy (#858)

* Tests/integration tests (#871)

* orb agent tags editing

* Feature/orb fleet increase unit tests coverage (#801)

* test(fleet): improvement on unit tests coverage of data access layer

* test(fleet): improvements on agent service unit tests coverage

* test(fleet): improvements on api unit tests coverage

* test(fleet): increase unit tests coverage of fleet service

* test(fleet): first unit test of comms service

* test(fleet): improvements on service and endpoint unit tests and minor corrections on endpoint response

* Test(fleet): unit tests for comms service

* test(fleet): add a mock nats PubSub

* edit agent name and tag scenarios (#872)

* Test/orb fleet unit test mix tags (#868)

* create scenarios to test mix of agent and orb tags

* create more test cases for matching agents

* new test case for unmatching tags

* fix expected return of matching agents feature on postgres's unit test (#874)

* fix broken unit tests on fleet agents (#877)

* Tests/editing group tags name description (#876)

* editing groups' tags, name and description

* Tests/updating documentation (#881)

* uptading test documentation

* Bug/agent group subscription fail (#857)

* wip solve agent_group bug editing tag process

* connect new matching agents to group channel

* fix policy apply after tags editing

* handle dataset when editing agent groups

* ignore conflicts on channel subscription

* fix unit tests fail

* create a fullList param to handle with agent policies

* refactor handleAgentPolicies (reducing cyclomaticy complexity)

* extract remove policy to a new method

* create loop to remove policies

* changes after code review (handling errors)

* feat(orb-ui): Agent View Enhancements (#856)

* wip

* fix(orb-ui): some css and page fixes removing unecessary pieces

* fix taps json output prettifier and text color
* remove unecessary links to manage and edit agent properties

* few more fixes and css

* feat(orb-ui):

* add policy state to header
* fix issues with header format

* fix(orb-ui): improvements to css overall

* fix(orb-ui): fix agent provisioning command key for agent view

* fix(orb-ui): update command provisioning card

* wip

* fix(orb-ui): fix observable chaining

* couple fixes on provisioning command icon

* fix reload and add agent group list with view and edit links

* css adjustments

* fix prefer interfaces over types

* fix css detail on agent group link

* fix(orb-ui): ironclad email issues (#888)

* wip

* fix code styling

* update node version

* update config for localhost https server for testing

* wip

* restore angular.json

* fix(orb-ui): agent group add page matching agents table status (#891)

* fix status color

* fix time format for last_hb

* code style fixes

* fix(orb-ui): agent group matching agents list issues (#893)

* wip

* fix time format for last_hb

* code style fixes

* wip

* fix bug when updating tags for agent groupings

* fix(orb-ui): register redirect issue (#898)

* wip - this return shouldn't be here and is certainly preventing the re-route and authentication

* wip

* on refresh page, _ps would become undefined

* fix(orb-ui): lint missing new line (#911)

* including advanced net, dhcp and dns policies to automated tests (#908)

* including advanced policies to automated tests

* Tests/tagging scenario (#910)

* tagging scenarios with smoke, sanity and fail

* fix(orb-ui): change provisioning command to be rendered as is. (#912)

* Bug/orb agent status (#884)

* wip solve agent_group bug editing tag process

* connect new matching agents to group channel

* fix policy apply after tags editing

* handle dataset when editing agent groups

* ignore conflicts on channel subscription

* fix unit tests fail

* create ticket to check agent state every 60 seconds

* wip graceful agent stop

* create function to check agent communication

* fix broken unit tests

* wip

* changes after code review

* call ticker stop on routine stopping

* fix(orb-ui): Datatable pagination and sorting issues (#913)

* query all sinks at once, filter client side, sort client side.
* dynamic pagination and virtual scroll enabled.
* column width NOT FLUID
* table height and width with better fluidness
* improve tag filter - use , or ; to split search
* fix time format to display timezone and 24hour format

* fix: remove extra line from provisioning command on agent view (#917)

* fix(orb-ui): fix correct prop tracking 'version' (#920)

* add N/A and tooltip for policies that do not have version property defined yet

* show correct property 'version' on agent policies list page

* fix(orb-ui): define custom tag comparator for ngx-datatables (#921)

* fix order by matching agents in agent group list page

* Bug missing owner_id redis sinker update (#918)

* fix redis message without owner id

* fix unit test

* fix(orb-ui): fix handler modal button positions (#924)

* fix issue #865 button order is wrong in handler modal

* policy editing (#922)

Edit policy name, description, handler, host_specification,  bpf_filter_expression, pcap_source, only_qname_suffix, only_rcode

* hotfix: fix agent list data retrieval (#925)

* fix(orb-ui): (#932)

* fix add metadata with fullName and company

* verify that _ps is available and if not, ignore calls

* fix: remote write url instead of remote host property title (#933)

* Feat/orb 851 unlink removed sink from dataset and inactivate if in case (#887)

* feat(dataset): invalidate dataset and unlink deleted sink

* feat(dataset): add unit tests to DeleteSinkFromDataset and InactivateDatasetBySinkID

* feat(dataset): fix dataset inactivate by sink ID

* feat(dataset): enhancement on invalidating datasets with no sinks linked

* feat(dataset): fix method name doesn't match in DeleteSinkFromAllDatasets metrics labels, fix method signature and name and add unit tests to inactivate dataset by id service

* feat(dataset): rename DeleteSinkFromAllDatasets to DeleteSinkFromAllDatasetsInternal method and InactivateDatasetById to InactivateDatasetByIdInternal method

* feat(dataset): fix order of InactivateDatasetByIDInternal arguments to be consistent with preexistings

* test(sinker): add more unit test for sinker parser (#873)

* test(sinker): add more unit tests for sinker parser

* test(sinker): add unit tests for DNS and packets parser

* test(sinker): add unit tests for rates and period parser

* fix page object for agent view (#934)

* check company and full name for registered accounts (#930)

* check company and full name for registered accounts

* Bug/orb fleet fail subscribe only one group (#902)

* fix subscription fail when agent has only one group

* wip

* avoid call two times policyreq

* make policy sanitization action more explicity

* Feat/orb broken agent reset (#929)

* Create logic to start broken backends

* Create logic do restart broken agent

* make pktvisor state handler on restart more explicity

* change view to get mix of agents (#915)

* change view to get mix of agents

* solve problem to connect to channel when using auto provisioning command

* code sanitization

* handle db transaction on failure

* Remove unnecessary transaction management and create migrations for view update

* fix(orb-ui): fix broken dataset details view (#939)

* method for retrieving sinks not updated after sinkService refactor.

* fix(policy): set version number to display zero after creating (#926)

* fix(orb-ui): fix agent view datasets showing in wrong policy nest (#943)

* fix(orb-ui): fix wrong conditional in version templatecell (#944)

* Tests/scenarios adjustments (#953)

* adjustments made to avoid false failure

* Set 0.13.0-develop release (#955)

* Feat/orb expose fleet metrics (#892)

* bootstraping instrumetation for comms service

* implement instumentation on comms for fleet

* wip

* create labels for the metrics to be exposed

* create instrumentation for comms method

* send operacional metrics from received payloads on sinker

* wip

* create metrics for sinker

* create a starter for https server and change some metrics names

* remove gauge from fleet comms

* fix after code review

* fix(orb-ui): agent policy list version number (#958)

* automated tests for sink removal (#963)

* automated tests for sink removal

* fix(orb-ui): (#962)

change provisioning command AGENT KEY string and
add informative text

* feat(orb-ui): yaml editor for policies (#935)

* editor online

* wip - improvements on css

* wip improving steps wizard/manual

* wip - should be able to add yaml policies - edit not implemented - may break

* fix: add backend:'pktvisor' to yaml submit payload

* wip update

* wip - moved toggle to second step

* Bug/orb fleet agent details (#966)

* change handle for offline heartbeats

* create new status for policies, groups and datasets on agent

* fix: add information text for total of policies (#973)

* fix dataset sink list display style (#972)

* fix: deletions no longer case insensitive (#971)

* Fix/unable to editing dataset name and sink name (#959)

* fix(dataset): allow editing dataset name

* fix(sinks): add name to update query and add retrieve by id after editing sink

* fix(orb-ui): fix wrong text on agent create provisioning command view. (#970)

* fix wrong text

* fix: remove unecessary text

* wip - fix for agent wizard second step not forwarding to third step (#969)

* fix(orb-ui): agent view issues (#968)

* wip - split page into components for each section/card

* wip - continuing refactor

* wip - css

* wip - rounding up components

* fix agent view

* fix lint

* fix(orb-ui): (#976)

* better error handling and more resilient agent view page

* fix(orb-ui): tab navigation issues (#975)

* wip

* wip

* wip - replaced tag control input in each page with newly created tag-control-component

* remove dead code

* sink page focus review

* refactor tag-control

* sink create page tab navigation

* wip

* add ngx-tag-display component
* review agent add/edit page navigation

* wip agent group and agent add and edit pages navigation review

* wip:

* review dataset add/edit navigation

* fix(orb-ui):
review all create/edit pages forms navigations

* fix(orb-ui): (#979)

* Fix code for matching tags mistakenly removed.

* Fix second step button type, now moving to third step correctly.

* feature(policy): add support for yaml formatted policy (#942)

* feature(policy): add support for yaml formatted policy

* feature(policy): fix yaml to json parser to follow collection policy top level keys

* feature(policy): fix unit test cases with policy_data

* feature(policy): update create, edit and view policy openapi spec

* feature(policy): -add column format to db and reflect  -refactor methods containing format and policy_data as parameters to get them from policy structure

* feature(policy): add policy_data and format to add, edit and valdate API responses

* fix(orb-ui): move click event listener to button container, parent of icon which was currently listening on click event (#1004)

* fix(orb-ui): add example of yaml policy to editor (#1003)

* fix(orb-ui): policy status (#1009)

* fix(orb-ui): agent view group listing css (#1010)

* fix(orb-ui): fix css and link style to agent group list items

* fix(orb-ui): button text decoration underline

* ui command quick fix (#1006)

* orb(ui): fix dataset details modal failed state (#1002)

* add message for non-retrievable entities - agent group, agent policy, sink;
* fix requests failing on error since forkjoin quits on error - adjust services accordingly

* Feat/orb sinker redis (#965)

* bootstraping sinker redis

* configure redis cache for sinker

* remove unnecessary class

* create unit test to redis cache

* wip

* implent return all configs from redis

* Fix unit tests

* Refactor sinker middleware (redis stream)

* enhancements and code sanitization

* fix sink idle state

* change alias for imported class

* enhacements os sinker cache (include ownerID)

* remove duplicated redis event for sink config editing process

* fix unit test

* refactor on sinker state checker

* fix broken unit test

* creation condition to only change state to idle from active configs

* remove idle configurations from Redis cache

* remove unncessary state verification

* Update git actions workflow (#992)

* [FIX] organizing git actions flow

* feat(orb-ui): #984 agent policy view (#1012)

* wip

* improve styling

* fix(orb-ui): fix yaml policy example (#1022)

* fix(orb-ui): #957 orb pagination (#1013)

* wip - sinks service fix

* wip - agent, agent groups, agent policies and datasets services

* general fixes to agent and policy view components (#1017)

* [FIX] add suffix when build develop agent (#1026)

* [FIX] add suffix on build and package jobs

* Fix prod flow (#1027)

* [FIX] production flow

* Feat/orb fleet matching groups (#978)

* Create matching groups for agents

* update openapi spec

* create unit test for matching agent groups

* create new endpoint to agent matching groups

* update open api spec

* Fix/950 validate dataset after editing (#977)

* fix(dataset): activate again dataset when adding a new sink after deleting one and suit unit tests

* fix(orb-ui): #997 agent policy yaml editing (#1032)

* fix(orb-ui): #997 agent policy yaml editing

* improve failed to fetch message (#1023)

* feat(orb-ui): added agent reset button (#1035)

* inserting simple policies to avoid false failure (#1036)

* fix(orb-ui): #362 lists are not properly refreshed after item deletion (#1046)

* hotfix(orb-ui): Change Agent Policy View page title (#1047)

* hotfix(orb-ui): change agent reset button placement (#1048)

* hotfix(orb-ui): improvements on agent view page css and error messages (#1050)

* Fix/1005 orb remove deleted group from dataset db (#1029)

* fix(dataset): remove agent group from dataset  db when it gets invalid because of group deletion

* implement auto backend restart on failure (#1062)

* implement auto backend restart on failure

* force proc stop, and ensure scraper stops too.

* feat(orb-ui): #985 agent policy groups (#1016)

* wip

* add agent groups section to policy view

* fix checking for failed policies during reapply by adding missing err check (#1066)

* fix(dataset): add service to remove policy from dataset when it gets invalid because of a policy deletion (#1063)

* Tests/reset agent remotely (#1042)

* reset agent remotely

* hotfix(orb-ui): quick and dirty fix to force clean cache after edit or delete of entity (#1061)

* force clean cache on every item operation.

* hotfix(orb-ui): remove unecessary code (#1065)

* removed dead code and deps

* remove dead code

* Add changes to production (#1028) (#1071)

* Revert "fix(dataset): add service to remove policy from dataset when it gets invalid because of a policy deletion (#1063)" (#1074)

This reverts commit 4edd705.

Co-authored-by: mclcavalcante <“[email protected]”>

* fix(policies) this fixes the double declaration of the methods

Co-authored-by: dscabral <[email protected]>
Co-authored-by: gpazuch <[email protected]>
Co-authored-by: dscabral <[email protected]>
Co-authored-by: Shannon Weyrick <[email protected]>
Co-authored-by: mclcavalcante <[email protected]>
Co-authored-by: manrodrigues <[email protected]>
Co-authored-by: Stéfano de Faria <[email protected]>
Co-authored-by: eric-daitan <[email protected]>
Co-authored-by: etaques <[email protected]>
Co-authored-by: mclcavalcante <“[email protected]”>
Co-authored-by: Luiz H Pegoraro <[email protected]>
  • Loading branch information
12 people authored Apr 19, 2022
1 parent f858423 commit 01afd2d
Show file tree
Hide file tree
Showing 190 changed files with 7,138 additions and 4,714 deletions.
30 changes: 30 additions & 0 deletions .github/workflows/agent.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,31 @@ jobs:
with:
go-version: 1.16

- name: Get branch name
shell: bash
run: echo "BRANCH_NAME=$(echo ${GITHUB_REF#refs/heads/} | tr / -)" >> $GITHUB_ENV

- name: Debug branch name
run: echo ${{ env.BRANCH_NAME }}

- name: Generate ref tag (main)
if: ${{ env.BRANCH_NAME == 'main' }}
run: |
echo "REF_TAG=latest" >> $GITHUB_ENV
- name: Generate ref tag (develop)
if: ${{ env.BRANCH_NAME == 'develop' }}
run: |
echo "REF_TAG=develop" >> $GITHUB_ENV
- name: Debug ref tag
run: echo ${{ env.REF_TAG }}

- name: Append suffix on VERSION file for develop build
if: ${{ env.BRANCH_NAME == 'develop' }}
run: |
echo "`cat ${{github.workspace}}/VERSION`-${{ env.REF_TAG }}" > VERSION
- name: Build orb agent (go build only)
run: make agent_bin

Expand Down Expand Up @@ -49,6 +74,11 @@ jobs:
- name: Debug ref tag
run: echo ${{ env.REF_TAG }}

- name: Append suffix on VERSION file for develop build
if: ${{ env.BRANCH_NAME == 'develop' }}
run: |
echo "`cat ${{github.workspace}}/VERSION`-${{ env.REF_TAG }}" > VERSION
- name: Get VERSION
run: |
echo "VERSION=`cat ${{github.workspace}}/VERSION`" >> $GITHUB_ENV
Expand Down
210 changes: 210 additions & 0 deletions .github/workflows/go-develop.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,210 @@
name: Go

on:
workflow_dispatch:
inputs:
pktvisor_tag:
description: 'pktvisor agent docker tag to package'
required: true
pull_request:
branches: [ develop ]
push:
branches: [ develop ]

jobs:

build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- uses: dorny/paths-filter@v2
id: filter
with:
filters: |
agent:
- 'agent/**'
- 'cmd/agent/**'
fleet:
- 'fleet/**'
- 'cmd/fleet/**'
policies:
- 'policies/**'
- 'cmd/policies/**'
sinks:
- 'sinks/**'
- 'cmd/sinks/**'
sinker:
- 'sinker/**'
- 'cmd/sinker/**'
ui:
- 'ui/**'
- name: Set up Go
uses: actions/setup-go@v2
with:
go-version: 1.16

- name: Set branch name
shell: bash
run: |
echo "BRANCH_NAME=develop" >> $GITHUB_ENV
- name: Generate ref tag (develop)
run: |
echo "REF_TAG=develop" >> $GITHUB_ENV
- name: Append suffix on VERSION file for develop build
run: |
echo "`cat ${{github.workspace}}/VERSION`-${{ env.REF_TAG }}" > VERSION
- name: Get VERSION
run: |
echo "VERSION=`cat ${{github.workspace}}/VERSION`" >> $GITHUB_ENV
- name: Debug VERSION
run: echo ${{ env.VERSION }}

- name: Build services (go build only)
run: make services

- name: Build orb agent (go build only)
if: steps.filter.outputs.agent == 'true'
run: make agent_bin

- name: Go unit tests
run: make test

- name: Build UI
if: steps.filter.outputs.ui == 'true'
run: make ui

package:
needs: build
runs-on: ubuntu-latest
# if this is a push into one of our main branches (rather than just a pull request), we will also package
if: github.event_name != 'pull_request'

steps:
- uses: actions/checkout@v2
- uses: dorny/paths-filter@v2
id: filter
with:
filters: |
agent:
- 'agent/**'
- 'cmd/agent/**'
fleet:
- 'fleet/**'
- 'cmd/fleet/**'
policies:
- 'policies/**'
- 'cmd/policies/**'
sinks:
- 'sinks/**'
- 'cmd/sinks/**'
sinker:
- 'sinker/**'
- 'cmd/sinker/**'
ui:
- 'ui/**'
- name: Set branch name
shell: bash
run: |
echo "BRANCH_NAME=develop" >> $GITHUB_ENV
- name: Generate ref tag (develop)
run: |
echo "REF_TAG=develop" >> $GITHUB_ENV
- name: Append suffix on VERSION file for develop build
run: |
echo "`cat ${{github.workspace}}/VERSION`-${{ env.REF_TAG }}" > VERSION
- name: Get VERSION
run: |
echo "VERSION=`cat ${{github.workspace}}/VERSION`" >> $GITHUB_ENV
- name: Debug VERSION
run: echo ${{ env.VERSION }}

- name: Get short commit hash to a variable
id: commit_hash
run: |
echo "::set-output name=sha_short::$(git rev-parse --short HEAD)"
- name: Commit orb-ui-live on orb-live repo develop
run: |
git config --global user.email "${{secrets.GH_ORB_EMAIL}}"
git config --global user.name "${{secrets.GH_ORB_USER}}"
git config --global credential.helper cache
git clone -b develop https://${{secrets.GH_ORB_USER}}:${{secrets.GH_ORB_ACCESS_TOKEN}}@github.com/ns1labs/orb-live.git
cd orb-live
cp -rf ../ui .
cp -rf ../VERSION .
echo "${{ steps.commit_hash.outputs.sha_short }}" > COMMIT_HASH
git add .
git commit -m "[NS1 Orb Bot] Update Orb Live UI for develop"
git push origin develop
- name: Login to Docker Hub
uses: docker/login-action@v1
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}

- name: Build service containers
run: make dockers

- name: Build orb-agent
shell: bash
if: steps.filter.outputs.agent == 'true'
env:
IMAGE_NAME: ns1labs/orb-agent
run: |
if [ "${{ github.event.inputs.pktvisor_tag }}" == "" ]; then
make agent
make agent_debug
else
PKTVISOR_TAG=${{ github.event.inputs.pktvisor_tag }} make agent
PKTVISOR_TAG=${{ github.event.inputs.pktvisor_tag }} make agent_debug
fi
- name: Push agent container
if: steps.filter.outputs.agent == 'true'
run: |
docker push -a ns1labs/orb-agent
- name: Build orb-ui
if: steps.filter.outputs.ui == 'true'
env:
IMAGE_NAME: ns1labs/orb-ui
run: |
make ui
- name: Push ui container
if: steps.filter.outputs.ui == 'true'
run: |
docker push -a ns1labs/orb-ui
- name: Push service containers
run: |
docker push -a ns1labs/orb-fleet
docker push -a ns1labs/orb-policies
docker push -a ns1labs/orb-sinks
docker push -a ns1labs/orb-sinker
- name: Commit image tag on orb-live-manifest environment dev
run: |
git config --global user.email "${{secrets.GH_ORB_EMAIL}}"
git config --global user.name "${{secrets.GH_ORB_USER}}"
git config --global credential.helper cache
git clone https://${{secrets.GH_ORB_USER}}:${{secrets.GH_ORB_ACCESS_TOKEN}}@github.com/ns1labs/orb-live-manifest.git
cd orb-live-manifest/stg
mv values.yaml .template/values.old
cat .template/values.yaml.tpl >> values.yaml
sed -i -e "s/IMAGE_TAG/${{ env.VERSION }}-${{ steps.commit_hash.outputs.sha_short }}/g" values.yaml
git add values.yaml
git add .template/values.old
git commit -m "[NS1 Orb Bot] Update image tag on stg environment"
git push origin main
Loading

0 comments on commit 01afd2d

Please sign in to comment.