desc: | Rasa Changelog |
---|
All notable changes to this project will be documented in this file. This project adheres to Semantic Versioning starting with version 1.0.
- Added data validator that checks if domain object returned is empty. If so, exit early
from the command
rasa data validate
. - Added the KeywordIntentClassifier.
- Added documentation for
AugmentedMemoizationPolicy
. - Fall back to
InMemoryTrackerStore
in case there is any problem with the current tracker store. - Arbitrary metadata can now be attached to any
Event
subclass. The data must be stored under themetadata
key when reading the event from a JSON object or dictionary. - Add command line argument
rasa x --config CONFIG
, to specify path to the policy and NLU pipeline configuration of your bot (default:config.yml
). - Added a new NLU featurizer -
ConveRTFeaturizer
based on ConveRT model released by PolyAI. - Added a new preconfigured pipeline -
pretrained_embeddings_convert
.
- Do not retrain the entire Core model if only the
templates
section of the domain is changed. - Upgraded
jsonschema
version.
- Remove duplicate messages when creating training data (issues/1446).
MultiProjectImporter
now imports files in the order of the import statements- Fixed server hanging forever on leaving
rasa shell
before first message - Fixed rasa init showing traceback error when user does Keyboard Interrupt before choosing a project path
CountVectorsFeaturizer
featurizes intents only if its analyzer is set toword
- Fixed bug where facebooks generic template was not rendered when buttons were
None
- Fixed default intents unnecessarily raising undefined parsing error
- Fixed Rasa X not working when any tracker store was configured for Rasa.
- Use the matplotlib backend
agg
in case thetkinter
package is not installed.
- NLU-only models no longer throw warnings about parsing features not defined in the domain
- Fixed bug that stopped Dockerfiles from building version 1.4.4.
- Fixed format guessing for e2e stories with intent restated as
/intent
PikaEventProducer
adds the RabbitMQApp ID
message property to published messages with the value of theRASA_ENVIRONMENT
environment variable. The message property will not be assigned if this environment variable isn't set.
- Updated Mattermost connector documentation to be more clear.
- Updated format strings to f-strings where appropriate.
- Updated tensorflow requirement to
1.15.0
- Dump domain using UTF-8 (to avoid
\UXXXX
sequences in the dumped files)
- Fixed exporting NLU training data in
json
format fromrasa interactive
- Fixed numpy deprecation warnings
- Fixed
Connection reset by peer
errors and bot response delays when using the RabbitMQ event broker.
- TensorFlow deprecation warnings are no longer shown when running
rasa x
- Fixed
'Namespace' object has no attribute 'persist_nlu_data'
error during interactive learning - Pinned networkx~=2.3.0 to fix visualization in rasa interactive and Rasa X
- Fixed
No model found
error when usingrasa run actions
with "actions" as a directory.
Regression: changes from 1.2.12
were missing from 1.4.0
, readded them
- add flag to CLI to persist NLU training data if needed
- log a warning if the
Interpreter
picks up an intent or an entity that does not exist in the domain file. - added
DynamoTrackerStore
to support persistence of agents running on AWS - added docstrings for
TrackerStore
classes - added buttons and images to mattermost.
CRFEntityExtractor
updated to accept arbitrary token-level features like word vectors (issues/4214)SpacyFeaturizer
updated to addner_features
forCRFEntityExtractor
- Sanitizing incoming messages from slack to remove slack formatting like <mailto:xyz@rasa.com|[email protected]> or <http://url.com|url.com> and substitute it with original content
- Added the ability to configure the number of Sanic worker processes in the HTTP
server (
rasa.server
) and input channel server (rasa.core.agent.handle_channels()
). The number of workers can be set using the environment variableSANIC_WORKERS
(default: 1). A value of >1 is allowed only in combination withRedisLockStore
as the lock store. - Botframework channel can handle uploaded files in
UserMessage
metadata. - Added data validator that checks there is no duplicated example data across multiples intents
- Unknown sections in markdown format (NLU data) are not ignored anymore, but instead an error is raised.
- It is now easier to add metadata to a
UserMessage
in existing channels. You can do so by overwriting the methodget_metadata
. The return value of this method will be passed to theUserMessage
object. - Tests can now be run in parallel
- Serialise
DialogueStateTracker
as json instead of pickle. DEPRECATION warning: Deserialisation of pickled trackers will be deprecated in version 2.0. For now, trackers are still loaded from pickle but will be dumped as json in any subsequent save operations. - Event brokers are now also passed to custom tracker stores (using the
event_broker
parameter) - Don't run the Rasa Docker image as
root
. - Use multi-stage builds to reduce the size of the Rasa Docker image.
- Updated the
/status
api route to use the actual model file location instead of thetmp
location.
- Removed Python 3.5 support
- fixed missing
tkinter
dependency for running tests on Ubuntu - fixed issue with
conversation
JSON serialization - fixed the hanging HTTP call with
ner_duckling_http
pipeline - fixed Interactive Learning intent payload messages saving in nlu files
- fixed DucklingHTTPExtractor dimensions by actually applying to the request
- Can now pass a package as an argument to the
--actions
parameter of therasa run actions
command.
- Fixed visualization of stories with entities which led to a failing visualization in Rasa X
- Port of 1.2.10 (support for RabbitMQ TLS authentication and
port
key in event broker endpoint config). - Port of 1.2.11 (support for passing a CA file for SSL certificate verification via the --ssl-ca-file flag).
- Fixed the hanging HTTP call with
ner_duckling_http
pipeline. - Fixed text processing of
intent
attribute insideCountVectorFeaturizer
. - Fixed
argument of type 'NoneType' is not iterable
when usingrasa shell
,rasa interactive
/rasa run
- Policies now only get imported if they are actually used. This removes TensorFlow warnings when starting Rasa X
- Fixed error
Object of type 'MaxHistoryTrackerFeaturizer' is not JSON serializable
when runningrasa train core
- Default channel
send_
methods no longer support kwargs as they caused issues in incompatible channels
- re-added TLS, SRV dependencies for PyMongo
- socketio can now be run without turning on the
--enable-api
flag - MappingPolicy no longer fails when the latest action doesn't have a policy
- Added the ability for users to specify a conversation id to send a message to when
using the
RasaChat
input channel.
- Fixed issue where
rasa init
would fail without spaCy being installed
- Added the ability to set the
backlog
parameter in Sanicsrun()
method using theSANIC_BACKLOG
environment variable. This parameter sets the number of unaccepted connections the server allows before refusing new connections. A default value of 100 is used if the variable is not set. - Status endpoint (
/status
) now also returns the number of training processes currently running
- Added the ability to properly deal with spaCy
Doc
-objects created on empty strings as discussed here. Only training samples that actually bear content are sent toself.nlp.pipe
for every given attribute. Non-content-bearing samples are converted to emptyDoc
-objects. The resulting lists are merged with their preserved order and properly returned. - asyncio warnings are now only printed if the callback takes more than 100ms (up from 1ms).
agent.load_model_from_server
no longer affects logging.
- The endpoint
POST /model/train
no longer supports specifying an output directory for the trained model using the fieldout
. Instead you can choose whether you want to save the trained model in the default model directory (models
) (default behavior) or in a temporary directory by specifying thesave_to_default_model_directory
field in the training request.
- Added a check to avoid training
CountVectorizer
for a particular attribute of a message if no text is provided for that attribute across the training data. - Default one-hot representation for label featurization inside
EmbeddingIntentClassifier
if label features don't exist. - Policy ensemble no longer incorrectly wrings "missing mapping policy" when mapping policy is present.
- "test" from
utter_custom_json
now correctly saved to tracker when using telegram channel
- Removed computation of
intent_spacy_doc
. As a result, none of the spacy components process intents now.
- SQL tracker events are retrieved ordered by timestamps. This fixes interactive learning events being shown in the wrong order.
- Pin gast to == 0.2.2
- Added option to persist nlu training data (default: False)
- option to save stories in e2e format for interactive learning
- bot messages contain the
timestamp
of theBotUttered
event, which can be used in channels FallbackPolicy
can now be configured to trigger when the difference between confidences of two predicted intents is too narrow- experimental training data importer which supports training with data of multiple sub bots. Please see the docs for more information.
- throw error during training when triggers are defined in the domain without
MappingPolicy
being present in the policy ensemble - The tracker is now available within the interpreter's
parse
method, giving the ability to create interpreter classes that use the tracker state (eg. slot values) during the parsing of the message. More details on motivation of this change see issues/3015. - add example bot
knowledgebasebot
to showcase the usage ofActionQueryKnowledgeBase
softmax
starspace loss for bothEmbeddingPolicy
andEmbeddingIntentClassifier
balanced
batching strategy for bothEmbeddingPolicy
andEmbeddingIntentClassifier
max_history
parameter forEmbeddingPolicy
- Successful predictions of the NER are written to a file if
--successes
is set when runningrasa test nlu
- Incorrect predictions of the NER are written to a file by default. You can disable it via
--no-errors
. - New NLU component
ResponseSelector
added for the task of response selection - Message data attribute can contain two more keys -
response_key
,response
depending on the training data - New action type implemented by
ActionRetrieveResponse
class and identified withresponse_
prefix - Vocabulary sharing inside
CountVectorsFeaturizer
withuse_shared_vocab
flag. If set to True, vocabulary of corpus is shared between text, intent and response attributes of message - Added an option to share the hidden layer weights of text input and label input inside
EmbeddingIntentClassifier
using the flagshare_hidden_layers
- New type of training data file in NLU which stores response phrases for response selection task.
- Add flag
intent_split_symbol
andintent_tokenization_flag
to allWhitespaceTokenizer
,JiebaTokenizer
andSpacyTokenizer
- Added evaluation for response selector. Creates a report
response_selection_report.json
inside--out
directory. - argument
--config-endpoint
to specify the URL from whichrasa x
pulls the runtime configuration (endpoints and credentials) LockStore
class storing instances ofTicketLock
for everyconversation_id
- environment variables
SQL_POOL_SIZE
(default: 50) andSQL_MAX_OVERFLOW
(default: 100) can be set to control the pool size and maximum pool overflow forSQLTrackerStore
when used with thepostgresql
dialect - Add a bot_challenge intent and a utter_iamabot action to all example projects and the rasa init bot.
- Allow sending attachments when using the socketio channel
rasa data validate
will fail with a non-zero exit code if validation fails
- added character-level
CountVectorsFeaturizer
with empirically found parameters into thesupervised_embeddings
NLU pipeline template - NLU evaluations now also stores its output in the output directory like the core evaluation
- show warning in case a default path is used instead of a provided, invalid path
- compare mode of
rasa train core
allows the whole core config comparison, naming style of models trained for comparison is changed (this is a breaking change) - pika keeps a single connection open, instead of open and closing on each incoming event
RasaChatInput
fetches the public key from the Rasa X API. The key is used to decode the bearer token containing the conversation ID. This requiresrasa-x>=0.20.2
.- more specific exception message when loading custom components depending on whether component's path or class name is invalid or can't be found in the global namespace
- change priorities so that the
MemoizationPolicy
has higher priority than theMappingPolicy
- substitute LSTM with Transformer in
EmbeddingPolicy
EmbeddingPolicy
can now useMaxHistoryTrackerFeaturizer
- non zero
evaluate_on_num_examples
inEmbeddingPolicy
andEmbeddingIntentClassifier
is the size of hold out validation set that is excluded from training data - defaults parameters and architectures for both
EmbeddingPolicy
andEmbeddingIntentClassifier
are changed (this is a breaking change) - evaluation of NER does not include 'no-entity' anymore
--successes
forrasa test nlu
is now boolean values. If set incorrect/successful predictions are saved in a file.--errors
is renamed to--no-errors
and is now a boolean value. By default incorrect predictions are saved in a file. If--no-errors
is set predictions are not written to a file.- Remove
label_tokenization_flag
andlabel_split_symbol
fromEmbeddingIntentClassifier
. Instead move these parameters toTokenizers
. - Process features of all attributes of a message, i.e. - text, intent and response inside the respective component itself. For e.g. - intent of a message is now tokenized inside the tokenizer itself.
- Deprecate
as_markdown
andas_json
in favour ofnlu_as_markdown
andnlu_as_json
respectively. - pin python-engineio >= 3.9.3
- update python-socketio req to >= 4.3.1
rasa test nlu
with a folder of configuration filesMappingPolicy
standard featurizer is set toNone
- Removed
text
parameter from send_attachment function in slack.py to avoid duplication of text output to slackbot - server
/status
endpoint reports status when an NLU-only model is loaded
- Removed
--report
argument fromrasa test nlu
. All output files are stored in the--out
directory.
- Support for transit encryption with Redis via
use_ssl: True
in the tracker store config in endpoints.yml
- Support for passing a CA file for SSL certificate verification via the --ssl-ca-file flag
- Added support for RabbitMQ TLS authentication. The following environment variables
need to be set:
RABBITMQ_SSL_CLIENT_CERTIFICATE
- path to the SSL client certificate (required)RABBITMQ_SSL_CLIENT_KEY
- path to the SSL client key (required)RABBITMQ_SSL_CA_FILE
- path to the SSL CA file (optional, for certificate verification)RABBITMQ_SSL_KEY_PASSWORD
- SSL private key password (optional) - Added ability to define the RabbitMQ port using the
port
key in theevent_broker
endpoint config.
- Correctly pass SSL flag values to x CLI command (backport of
- SQL tracker events are retrieved ordered by timestamps. This fixes interactive
learning events being shown in the wrong order. Backport of
1.3.2
patch (PR #4427).
- Added
query
dictionary argument toSQLTrackerStore
which will be appended to the SQL connection URL as query parameters.
- fixed bug that occurred when sending template
elements
through a channel that doesn't support them
- SSL support for
rasa run
command. Certificate can be specified using--ssl-certificate
and--ssl-keyfile
.
- made default augmentation value consistent across repo
'/restart'
will now also restart the bot if the tracker is paused
- the
SocketIO
input channel now allows accesses from other origins (fixesSocketIO
channel on Rasa X)
- messages with multiple entities are now handled properly with e2e evaluation
data/test_evaluations/end_to_end_story.md
was re-written in the restaurantbot domain
- messages with multiple entities are now handled properly with e2e evaluation
data/test_evaluations/end_to_end_story.md
was re-written in the restaurantbot domain
- Free text input was not allowed in the Rasa shell when the response template contained buttons, which has now been fixed.
UserUttered
events always got the same timestamp
- Docs now have an
EDIT THIS PAGE
button
Flood control exceeded
error in Telegram connector which happened because the webhook was set twice
- add root route to server started without
--enable-api
parameter - add
--evaluate-model-directory
torasa test core
to evaluate models fromrasa train core -c <config-1> <config-2>
- option to send messages to the user by calling
POST /conversations/{conversation_id}/execute
Agent.update_model()
andAgent.handle_message()
now work without needing to set a domain or a policy ensemble- Update pytype to
2019.7.11
- new event broker class:
SQLProducer
. This event broker is now used when running locally with Rasa X - API requests are not longer logged to
rasa_core.log
by default in order to avoid problems when running on OpenShift (use--log-file rasa_core.log
to retain the old behavior) metadata
attribute added toUserMessage
rasa test core
can handle compressed model files- rasa can handle story files containing multi line comments
- template will retain { if escaped with {. e.g. {{"foo": {bar}}} will result in {"foo": "replaced value"}
TrainingFileImporter
interface to support customizing the process of loading training data- fill slots for custom templates
Agent.update_model()
andAgent.handle_message()
now work without needing to set a domain or a policy ensemble- update pytype to
2019.7.11
- interactive learning bug where reverted user utterances were dumped to training data
- added timeout to terminal input channel to avoid freezing input in case of server errors
- fill slots for image, buttons, quick_replies and attachments in templates
rasa train core
in comparison mode stores the model files compressed (tar.gz
files)- slot setting in interactive learning with the TwoStageFallbackPolicy
- added optional pymongo dependencies
[tls, srv]
torequirements.txt
for better mongodb support case_sensitive
option added toWhiteSpaceTokenizer
withtrue
as default.
- validation no longer throws an error during interactive learning
- fixed wrong cleaning of
use_entities
in case it was a list and notTrue
- updated the server endpoint
/model/parse
to handle also messages with the intent prefix - fixed bug where "No model found" message appeared after successfully running the bot
- debug logs now print to
rasa_core.log
when runningrasa x -vv
orrasa run -vv
- rest channel supports setting a message's input_channel through a field
input_channel
in the request body
- recommended syntax for empty
use_entities
andignore_entities
in the domain file has been updated fromFalse
orNone
to an empty list ([]
)
rasa run
without--enable-api
does not require a local model anymore- using
rasa run
with--enable-api
to run a server now prints "running Rasa server" instead of "running Rasa Core server" - actions, intents, and utterances created in
rasa interactive
can no longer be empty
- debug logging now tells you which tracker store is connected
- the response of
/model/train
now includes a response header for the trained model filename Validator
class to help developing by checking if the files have any errors- project's code is now linted using flake8
info
log when credentials were provided for multiple channels and channel in--connector
argument was specified at the same time- validate export paths in interactive learning
- deprecate
rasa.core.agent.handle_channels(...)`. Please use ``rasa.run(...)
orrasa.core.run.configure_app
instead. Agent.load()
also acceptstar.gz
model file
- revert the stripping of trailing slashes in endpoint URLs since this can lead to problems in case the trailing slash is actually wanted
- starter packs were removed from Github and are therefore no longer tested by Travis script
- all temporal model files are now deleted after stopping the Rasa server
rasa shell nlu
now outputs unicode characters instead of\uxxxx
codes- fixed PUT /model with model_server by deserializing the model_server to EndpointConfig.
x in AnySlotDict
is nowTrue
for anyx
, which fixes empty slot warnings in interactive learningrasa train
now also includes NLU files in other formats than the Rasa formatrasa train core
no longer crashes without a--domain
argrasa interactive
now looks for endpoints inendpoints.yml
if no--endpoints
arg is passed- custom files, e.g. custom components and channels, load correctly when using the command line interface
MappingPolicy
now works correctly when used as part of a PolicyEnsemble
- unfeaturize single entities
- added agent readiness check to the
/status
resource
- removed leading underscore from name of '_create_initial_project' function.
- fixed bug where facebook quick replies were not rendering
- take FB quick reply payload rather than text as input
- fixed bug where training_data path in metadata.json was an absolute path
- fixed any inconsistent type annotations in code and some bugs revealed by type checker
- fixed duplicate events appearing in tracker when using a PostgreSQL tracker store
- fixed compatibility with Rasa SDK
- bot responses can contain
custom
messages besides other message types
- nlu configs can now be directly compared for performance on a dataset
in
rasa test nlu
- update the tracker in interactive learning through reverting and appending events instead of replacing the tracker
POST /conversations/{conversation_id}/tracker/events
supports a list of events
- fixed creation of
RasaNLUHttpInterpreter
- form actions are included in domain warnings
- default actions, which are overriden by custom actions and are listed in the domain are excluded from domain warnings
- SQL
data
column type toText
for compatibility with MySQL - non-featurizer training parameters don't break SklearnPolicy anymore
- revert PR #3739 (as this is a breaking change): set
PikaProducer
andKafkaProducer
default queues back torasa_core_events
- support for specifying full database urls in the
SQLTrackerStore
configuration - maximum number of predictions can be set via the environment variable
MAX_NUMBER_OF_PREDICTIONS
(default is 10)
- default
PikaProducer
andKafkaProducer
queues torasa_production_events
- exclude unfeaturized slots from domain warnings
- loading of additional training data with the
SkillSelector
- strip trailing slashes in endpoint URLs
- added argument
--rasa-x-port
to specify the port of Rasa X when running Rasa X locally viarasa x
- slack notifications from bots correctly render text
- fixed usage of
--log-file
argument forrasa run
andrasa shell
- check if correct tracker store is configured in local mode
- fixed backwards incompatible utils changes
- fixed spacy being a required dependency (regression)
- automatic creation of index on the
sender_id
column when using an SQL tracker store. If you have an existing data and you are running into performance issues, please make sure to add an index manually usingCREATE INDEX event_idx_sender_id ON events (sender_id);
.
- NLU evaluation in cross-validation mode now also provides intent/entity reports, confusion matrix, etc.
- non-ascii characters render correctly in stories generated from interactive learning
- validate domain file before usage, e.g. print proper error messages if domain file is invalid instead of raising errors
- added
domain_warnings()
method toDomain
which returns a dict containing the diff between supplied {actions, intents, entities, slots} and what's contained in the domain
- fix lookup table files failed to load issues/3622
- buttons can now be properly selected during cmdline chat or when in interactive learning
- set slots correctly when events are added through the API
- mapping policy no longer ignores NLU threshold
- mapping policy priority is correctly persisted
- updated installation command in docs for Rasa X
- added arguments to set the file paths for interactive training
- added quick reply representation for command-line output
- added option to specify custom button type for Facebook buttons
- added tracker store persisting trackers into a SQL database
(
SQLTrackerStore
) - added rasa command line interface and API
- Rasa HTTP training endpoint at
POST /jobs
. This endpoint will train a combined Rasa Core and NLU model ReminderCancelled(action_name)
event to cancel given action_name reminder for current user- Rasa HTTP intent evaluation endpoint at
POST /intentEvaluation
. This endpoints performs an intent evaluation of a Rasa model - option to create template for new utterance action in
interactive learning
- you can now choose actions previously created in the same session
in
interactive learning
- add formatter 'black'
- channel-specific utterances via the
- "channel":
key in utterance templates - arbitrary json messages via the
- "custom":
key in utterance templates and viautter_custom_json()
method in custom actions - support to load sub skills (domain, stories, nlu data)
- support to select which sub skills to load through
import
section inconfig.yml
- support for spaCy 2.1
- a model for an agent can now also be loaded from a remote storage
- log level can be set via environment variable
LOG_LEVEL
- add
--store-uncompressed
to train command to not compress Rasa model - log level of libraries, such as tensorflow, can be set via environment variable
LOG_LEVEL_LIBRARIES
- if no spaCy model is linked upon building a spaCy pipeline, an appropriate error message is now raised with instructions for linking one
- renamed all CLI parameters containing any
_
to use dashes-
instead (GNU standard) - renamed
rasa_core
package torasa.core
- for interactive learning only include manually annotated and ner_crf entities in nlu export
- made
message_id
an additional argument tointerpreter.parse
- changed removing punctuation logic in
WhitespaceTokenizer
training_processes
in the Rasa NLU data router have been renamed toworker_processes
- created a common utils package
rasa.utils
for nlu and core, common methods likeread_yaml
moved there - removed
--num_threads
from run command (server will be asynchronous but running in a single thread) - the
_check_token()
method inRasaChat
now authenticates against/auth/verify
instead of/user
- removed
--pre_load
from run command (Rasa NLU server will just have a maximum of one model and that model will be loaded by default) - changed file format of a stored trained model from the Rasa NLU server to
tar.gz
- train command uses fallback config if an invalid config is given
- test command now compares multiple models if a list of model files is provided for the argument
--model
- Merged rasa.core and rasa.nlu server into a single server. See swagger file in
docs/_static/spec/server.yaml
for available endpoints. utter_custom_message()
method in rasa_core_sdk has been renamed toutter_elements()
- updated dependencies. as part of this, models for spacy need to be reinstalled for 2.1 (from 2.0)
- make sure all command line arguments for
rasa test
andrasa interactive
are actually used, removed arguments that were not used at all (e.g.--core
forrasa test
)
- removed possibility to execute
python -m rasa_core.train
etc. (e.g. scripts inrasa.core
andrasa.nlu
). Use the CLI for rasa instead, e.g.rasa train core
. - removed
_sklearn_numpy_warning_fix
from theSklearnIntentClassifier
- removed
Dispatcher
class from core - removed projects: the Rasa NLU server now has a maximum of one model at a time loaded.
- evaluating core stories with two stage fallback gave an error, trying to handle None for a policy
- the
/evaluate
route for the Rasa NLU server now runs evaluation in a parallel process, which prevents the currently loaded model unloading - added missing implementation of the
keys()
function for the Redis Tracker Store - in interactive learning: only updates entity values if user changes annotation
- log options from the command line interface are applied (they overwrite the environment variable)
- all message arguments (kwargs in dispatcher.utter methods, as well as template args) are now sent through to output channels
- utterance templates defined in actions are checked for existence upon training a new agent, and a warning is thrown before training if one is missing