Merge pull request #596 from creative-commoners/pulls/6/field-validators

DOC Validate DBFields

en/02_Developer_Guides/00_Model/04_Data_Types_and_Casting.md

@@ -31,33 +31,41 @@ class Player extends DataObject
 }
 ```
 
+Most `DBField` subclasses will be validated using a [`FieldValidator`](api:SilverStripe\Core\Validation\FieldValidation\FieldValidator) subclass which is call as part of the `DataObject::validate()` method. This means that when a value is set on a `DBField` subclass, it will be validated against the constraints of that field. If the value is invalid then a [`ValidationException`](api:SilverStripe\Core\Validation\ValidationException) will be thrown.
+
 ## Available types
 
-- `'BigInt'`: An 8-byte signed integer field (see: [DBBigInt](api:SilverStripe\ORM\FieldType\DBBigInt)).
-- `'Boolean'`: A boolean field (see: [DBBoolean](api:SilverStripe\ORM\FieldType\DBBoolean)).
-- `'Currency'`: A number with 2 decimal points of precision, designed to store currency values. Only supports single currencies (see: [DBCurrency](api:SilverStripe\ORM\FieldType\DBCurrency)).
-- `'Date'`: A date field (see: [DBDate](api:SilverStripe\ORM\FieldType\DBDate)).
-- `'Datetime'`: A date/time field (see: [DBDatetime](api:SilverStripe\ORM\FieldType\DBDatetime)).
-- `'DBClassName'`: A special enumeration for storing class names (see: [DBClassName](api:SilverStripe\ORM\FieldType\DBClassName)).
-- `'Decimal'`: A decimal number (see: [DBDecimal](api:SilverStripe\ORM\FieldType\DBDecimal)).
-- `'Double'`: A floating point number with double precision (see: [DBDouble](api:SilverStripe\ORM\FieldType\DBDouble)).
-- `'Enum'`: An enumeration of a set of strings that can store a single value (see: [DBEnum](api:SilverStripe\ORM\FieldType\DBEnum)).
-- `'Float'`: A floating point number (see: [DBFloat](api:SilverStripe\ORM\FieldType\DBFloat)).
-- `'Foreignkey'`: A special `Int` field used for foreign keys in `has_one` relationships (see: [DBForeignKey](api:SilverStripe\ORM\FieldType\DBForeignKey)).
-- `'HTMLFragment'`: A variable-length string of up to 2MB, designed to store HTML. Doesn't process [shortcodes](/developer_guides/extending/shortcodes/). (see: [DBHTMLText](api:SilverStripe\ORM\FieldType\DBHTMLText)).
-- `'HTMLText'`: A variable-length string of up to 2MB, designed to store HTML. Processes [shortcodes](/developer_guides/extending/shortcodes/). (see: [DBHTMLText](api:SilverStripe\ORM\FieldType\DBHTMLText)).
-- `'HTMLVarchar'`: A variable-length string of up to 255 characters, designed to store HTML. Can process [shortcodes](/developer_guides/extending/shortcodes/) with additional configuration. (see: [DBHTMLVarchar](api:SilverStripe\ORM\FieldType\DBHTMLVarchar)).
-- `'Int'`: A 32-bit signed integer field (see: [DBInt](api:SilverStripe\ORM\FieldType\DBInt)).
-- `'Locale'`: A field for storing locales (see: [DBLocale](api:SilverStripe\ORM\FieldType\DBLocale)).
-- `'Money'`: Similar to Currency, but with localisation support (see: [DBMoney](api:SilverStripe\ORM\FieldType\DBMoney)).
-- `'MultiEnum'`: An enumeration set of strings that can store multiple values (see: [DBMultiEnum](api:SilverStripe\ORM\FieldType\DBMultiEnum)).
-- `'Percentage'`: A decimal number between 0 and 1 that represents a percentage (see: [DBPercentage](api:SilverStripe\ORM\FieldType\DBPercentage)).
-- `'PolymorphicForeignKey'`: A special ForeignKey class that handles relations with arbitrary class types (see: [DBPolymorphicForeignKey](api:SilverStripe\ORM\FieldType\DBPolymorphicForeignKey)).
-- `'PrimaryKey'`: A special type Int field used for primary keys. (see: [DBPrimaryKey](api:SilverStripe\ORM\FieldType\DBPrimaryKey)).
-- `'Text'`: A variable-length string of up to 2MB, designed to store raw text (see: [DBText](api:SilverStripe\ORM\FieldType\DBText)).
-- `'Time'`: A time field (see: [DBTime](api:SilverStripe\ORM\FieldType\DBTime)).
-- `'Varchar'`: A variable-length string of up to 255 characters, designed to store raw text (see: [DBVarchar](api:SilverStripe\ORM\FieldType\DBVarchar)).
-- `'Year'`: Represents a single year field (see: [DBYear](api:SilverStripe\ORM\FieldType\DBYear)).
+| Field | API link | Description | Validation |
+| --- | --- | --- | --- |
+| `BigInt` | [`DBBigInt`](api:SilverStripe\ORM\FieldType\DBBigInt) | An 8-byte signed integer field | Must be an int between -9223372036854775808 and 9223372036854775807 |
+| `Boolean` | [`DBBoolean`](api:SilverStripe\ORM\FieldType\DBBoolean) | A boolean field stored as a tinyint | Must be a boolean |
+| `Currency` | [`DBCurrency`](api:SilverStripe\ORM\FieldType\DBCurrency) | A number with 2 decimal points of precision, designed to store currency values | Must be a decimal |
+| `Date` | [`DBDate`](api:SilverStripe\ORM\FieldType\DBDate) | A date field | Must be a valid date in `Y-m-d` format |
+| `Datetime` | [`DBDatetime`](api:SilverStripe\ORM\FieldType\DBDatetime) | A date/time field | Must be a valid datetime in `Y-m-d H:i:s` format |
+| `ClassName` | [`DBClassName`](api:SilverStripe\ORM\FieldType\DBClassName) | A special enumeration for storing class names | Must be a valid FQCN of a `DataObject` subclass |
+| `ClassNameVarchar` | [`DBClassNameVarchar`](api:SilverStripe\ORM\FieldType\DBClassNameVarchar) | A special enumeration for storing class names in a `Varchar` field | Must be a valid FQCN of a `DataObject` subclass |
+| `Decimal` | [`DBDecimal`](api:SilverStripe\ORM\FieldType\DBDecimal) | A decimal number | Must be a decimal. |
+| `Double` | [`DBDouble`](api:SilverStripe\ORM\FieldType\DBDouble) | A floating point number with double precision | Must be numeric |
+| `Email` | [`DBEmail`](api:SilverStripe\ORM\FieldType\DBEmail) | An email field | Must be a valid email address |
+| `Enum` | [`DBEnum`](api:SilverStripe\ORM\FieldType\DBEnum) | An enumeration of a set of strings that can store a single value | Must be one of the defined values |
+| `Float` | [`DBFloat`](api:SilverStripe\ORM\FieldType\DBFloat) | A floating point number | Must be numeric |
+| `ForeignKey` | [`DBForeignKey`](api:SilverStripe\ORM\FieldType\DBForeignKey) | A special `Int` field used for foreign keys in `has_one` relationships | Must be an int |
+| `HTMLFragment` | [`DBHTMLText`](api:SilverStripe\ORM\FieldType\DBHTMLText) | A variable-length string of up to 2MB, designed to store HTML. Doesn't process [shortcodes](/developer_guides/extending/shortcodes/) | Must be a string |
+| `HTMLText` | [`DBHTMLText`](api:SilverStripe\ORM\FieldType\DBHTMLText) | A variable-length string of up to 2MB, designed to store HTML. Processes [shortcodes](/developer_guides/extending/shortcodes/) | Must be a string |
+| `HTMLVarchar` | [`DBHTMLVarchar`](api:SilverStripe\ORM\FieldType\DBHTMLVarchar) | A variable-length string of up to 255 characters, designed to store HTML. Can process [shortcodes](/developer_guides/extending/shortcodes/) with additional configuration | String must not be longer than specified length |
+| `Int` | [`DBInt`](api:SilverStripe\ORM\FieldType\DBInt) | A 32-bit signed integer field | Must be an int between -2147483648 and 2147483647 |
+| `IP` | [`DBIp`](api:SilverStripe\ORM\FieldType\DBIp) | An IP field | Must be a valid IP address, either IPv4 or IPv6 |
+| `Locale` | [`DBLocale`](api:SilverStripe\ORM\FieldType\DBLocale) | A field for storing locales | Must be a valid locale |
+| `Money` | [`DBMoney`](api:SilverStripe\ORM\FieldType\DBMoney) | A localised money field with a 3 character currency component, and an amount component. | Currency string must not be greater than 3 characters, amount must be a decimal |
+| `MutliEnum` | [`DBMultiEnum`](api:SilverStripe\ORM\FieldType\DBMultiEnum) | An enumeration set of strings that can store multiple values | Must be one of the allowable values |
+| `Percentage` | [`DBPercentage`](api:SilverStripe\ORM\FieldType\DBPercentage) | A decimal number between 0 and 1 that represents a percentage | Must be a decimal between 0 and 1 |
+| `PolymorphicForeignKey` | [`DBPolymorphicForeignKey`](api:SilverStripe\ORM\FieldType\DBPolymorphicForeignKey) | A special ForeignKey class that handles relations with arbitrary class types | Must be an int |
+| `PrimaryKey` | [`DBPrimaryKey`](api:SilverStripe\ORM\FieldType\DBPrimaryKey) | A special type Int field used for primary keys | Must be an int |
+| `Text` | [`DBText`](api:SilverStripe\ORM\FieldType\DBText) | A variable-length string of up to 2MB, designed to store raw text | Must be a string |
+| `Time` | [`DBTime`](api:SilverStripe\ORM\FieldType\DBTime) | A time field | Must be a valid time in `H:i:s` format |
+| `URL` | [`DBUrl`](api:SilverStripe\ORM\FieldType\DBUrl) | A URL field | Must be a valid URL |
+| `Varchar` | [`DBVarchar`](api:SilverStripe\ORM\FieldType\DBVarchar) | A variable-length string of up to 255 characters, designed to store raw text | String must not be longer than specified length |
+| `Year` | [`DBYear`](api:SilverStripe\ORM\FieldType\DBYear) | Represents a single year field | Must be a valid year between 1901 and 2155 |
 
 See the [API documentation](api:SilverStripe\ORM\FieldType) for a full list of available data types. You can define your own [`DBField`](api:SilverStripe\ORM\FieldType\DBField) instances if required as well.
 

en/02_Developer_Guides/00_Model/09_Validation.md

@@ -39,16 +39,11 @@ Traditionally, validation in Silverstripe CMS has been mostly handled through [f
 
 Most validation constraints are actually data constraints which belong on the model. Silverstripe CMS provides the
 [`DataObject::validate()`](api:SilverStripe\ORM\DataObject::validate()) method for this purpose. The `validate()` method is
-called any time the `write()` method is called, before the `onBeforeWrite()` extension hook.
-
-By default, there is no validation - objects are always valid! However, you can override this method in your `DataObject`
+called any time the `write()` method is called. Implement this method in your `DataObject`
 sub-classes to specify custom validation, or use the `updateValidate()` extension hook through an [Extension](api:SilverStripe\Core\Extension).
 
 Invalid objects won't be able to be written - a [`ValidationException`](api:SilverStripe\Core\Validation\ValidationException) will be thrown and no write will occur.
 
-Ideally you should call `validate()` in your own application to test that an object is valid before attempting a
-write, and respond appropriately if it isn't.
-
 The return value of `validate()` is a [`ValidationResult`](api:SilverStripe\Core\Validation\ValidationResult) object.
 
 ```php
@@ -82,6 +77,16 @@ class MyObject extends DataObject
 }
 ```
 
+## DBField validation
+
+[`DBField`](api:SilverStripe\ORM\FieldType\DBField) is the base class for all database fields in Silverstripe CMS. For instance when you defined `'MyField' => 'Varchar(255)'` in your [`DataObject`](api:SilverStripe\ORM\DataObject) subclass, the `MyField` property would be an instance of [`DBVarchar`](api:SilverStripe\ORM\FieldType\DBVarchar).
+
+Most `DBField` subclasses will have their values validated as part of `DataObject::validate()`. This means that when a value is set on a `DBField` subclass, it will be validated against the constraints of that field. Field validation is called as part of `DataObject::validate()` which itself is called as part of [`DataObject::write()`](api:SilverStripe\ORM\DataObject::write()). If a value is invalid then a [`ValidationException`](api:SilverStripe\ORM\Validation\ValidationException) will be thrown.
+
+For example, if you have a `Varchar(64)`, and you try to set a value longer than 64 characters, a validation exception will be thrown.
+
+A full list of DBField subclasses and their validation rules can be found in [Data Types and Casting](data_types_and_casting).
+
 ## API documentation
 
 - [DataObject](api:SilverStripe\ORM\DataObject)

en/08_Changelogs/6.0.0.md

@@ -7,6 +7,7 @@ title: 6.0.0 (unreleased)
 ## Overview
 
 - [Features and enhancements](#features-and-enhancements)
+  - [Validation added to DBFields](#dbfield-validation)
   - [Changes to `sake`, `BuildTask`, CLI interaction in general](#cli-changes)
   - [Read-only replica database support](#db-read-only-replicas)
   - [Run `CanonicalURLMiddleware` in all environments by default](#url-middleware)
@@ -39,6 +40,67 @@ title: 6.0.0 (unreleased)
 
 ## Features and enhancements
 
+### Validation added to `DBField`s {#dbfield-validation}
+
+[`DBField`](api:SilverStripe\ORM\FieldType\DBField) is the base class for all database fields in Silverstripe CMS. For instance when you defined `'MyField' => 'Varchar(255)'` in your [`DataObject`](api:SilverStripe\ORM\DataObject) subclass, the `MyField` property would be an instance of [`DBVarchar`](api:SilverStripe\ORM\FieldType\DBVarchar).
+
+Validation has added been to most `DBField` subclasses. This means that when a value is set on a `DBField` subclass, it will be validated against the constraints of that field. This field validation is called as part of [`DataObject::validate()`](api:SilverStripe\ORM\DataObject::validate()) which itself is called as part of [`DataObject::write()`](api:SilverStripe\ORM\DataObject::write()). If a value is invalid then a [`ValidationException`](api:SilverStripe\ORM\Validation\ValidationException) will be thrown.
+
+For example, if you have a `Varchar(64)`, and you try to set a value longer than 64 characters, a exception will now be thrown. Previously, the value would be truncated to 64 characters and saved to the database.
+
+The validation is added through subclasses of the new [`FieldValidator`](api:SilverStripe\Core\Validation\FieldValidation\FieldValidator) abstract class, for instance the [`StringFieldValidator`](api:SilverStripe\Core\Validation\FieldValidation\StringFieldValidator) is used to validate [`DBVarchar`](api:SilverStripe\ORM\FieldType\DBVarchar).
+
+Note that this new `DBField` validation is independent of the existing CMS form field validation that uses methods such as [`FormField::validate()`](api:SilverStripe\Forms\FormField::validate()) and [`DataObject::getCMSCompositeValidator()`](api:SilverStripe\ORM\DataObject::getCMSCompositeValidator()).
+
+Updates have been made to some `setValue()` methods of `DBField` subclasses to convert the value to the correct type before validating it.
+
+- [`DBBoolean`](api:SilverStripe\ORM\FieldType\DBBoolean) uses the `tinyint` data type and retains the legacy behaviour of converting `true/'true'/'t'/'1'` to `1`, `false/'false'/'f'/'0'` to `0`.
+- [`DBDecimal`](api:SilverStripe\ORM\FieldType\DBDecimal) will convert numeric strings as well as integers to floats.
+- [`DBForeignKey`](api:SilverStripe\ORM\FieldType\DBForeignKey) will convert a blank string to 0.
+- [`DBInt`](api:SilverStripe\ORM\FieldType\DBInt) will convert integer like strings to integers.
+- [`DBYear`](api:SilverStripe\ORM\FieldType\DBYear) will convert integer like strings to integers. Also shorthand years are converted to full years (e.g. "24" becomes "2024").
+
+In most cases though, the correct scalar type must now be used. For instance it is no longer possible to set an integer value on a `DBVarchar` field. You must use a string.
+
+Some new DBField subclasses have been added which will provide validation for specific types of data:
+
+- [`DBEmail`](api:SilverStripe\ORM\FieldType\DBEmail) for email addresses.
+- [`DBIp`](api:SilverStripe\ORM\FieldType\DBIp) for IP addresses.
+- [`DBUrl`](api:SilverStripe\ORM\FieldType\DBUrl) for URLs.
+
+To use these new field types, simply define them in a DataObject subclass:
+
+```php
+// app/src/Pages/MyPage.php
+namespace App\Pages;
+
+use SilverStripe\CMS\Model\SiteTree;
+
+class MyPage extends SiteTree
+{
+    private static array $db = [
+        // Values will be validated as an email address
+        'MyEmail' => 'Email',
+        // Values will be validated as an IP address
+        'MyIP' => 'IP',
+        // Values will be validated as a URL
+        'MyURL' => 'URL',
+    ];
+}
+```
+
+> [!WARNING]
+> If you have an existing project that uses Varchar fields for email addresses, IP addresses, or URLs, and you switch to using one of the new DBField types, be aware that some of the values in the database may fail validation the next time they are saved which could cause issues for CMS editors.
+> You may wish to create a [`BuildTask`](api:SilverStripe\Dev\BuildTask) that calls [`DataObject::validate()`](api:SilverStripe\ORM\DataObject::validate()) on all affected records.
+
+While we have tried to match the validation rules up to what would already have been stored in the database, there is a chance you'll find yourself with pre-existing data which doesn't meet the validation rules, and which therefore causes validation exceptions to be thrown the next time you try to save those records.
+
+If that happens, you may be able to resolve it with one of the following solutions:
+
+- If there is a form field for that database column, update the value in the form to a valid value before saving the record.
+- Write a `BuildTask` that updates any invalid values in your database.
+- While it's generally not recommended, you have the option of disabling validation via the [`DataObject.validation_enabled`](api:SilverStripe\ORM\DataObject->validation_enabled) configuration property.
+
 ### Changes to `sake`, `BuildTask`, and CLI interaction in general {#cli-changes}
 
 Until now, running `sake` on the command line has executed a simulated HTTP request to your Silverstripe CMS project, using the routing and controllers that your web application uses to handle HTTP requests. This resulted in both a non-standard CLI experience and added confusion about when an [`HTTPRequest`](api:SilverStripe\Control\HTTPRequest) actually represented an HTTP request.
@@ -811,6 +873,7 @@ As part of these changes [`ArrayList::find()`](api:SilverStripe\Model\List\Array
 - [`FieldList`](api:SilverStripe\Forms\FieldList) is now strongly typed. Methods that previously allowed any iterable variables to be passed, namely [`FieldList::addFieldsToTab()`](api:SilverStripe\Forms\FieldList::addFieldsToTab()) and [`FieldList::removeFieldsFromTab()`](api:SilverStripe\Forms\FieldList::removeFieldsFromTab()), now require an array to be passed instead.
 - [`BaseElement::getDescription()`](api:DNADesign\Elemental\Models\BaseElement::getDescription()) has been removed. If you had implemented this method in your custom elemental blocks, either set the [`description`](api:DNADesign\Elemental\Models\BaseElement->description) configuration property or override the [`getTypeNice()`](api:DNADesign\Elemental\Models\BaseElement::getTypeNice()) method.
 - [`DataExtension`](api:SilverStripe\ORM\DataExtension), [`SiteTreeExtension`](api:SilverStripe\CMS\Model\SiteTreeExtension), and [`LeftAndMainExtension`](api:SilverStripe\Admin\LeftAndMainExtension) have been removed. If you subclass any of these classes, you must now subclass [`Extension`](api:SilverStripe\Core\Extension) instead.
+- [`DBCurrency`](api:SilverStripe\ORM\FieldType\DBCurrency) will no longer parse numeric values contained in a string when calling `setValue()`. For instance "this is 50.29 dollars" will no longer be converted to "$50.29", instead its value will remain as "this is 50.29 dollars" and it will throw a validation exception if attempted to be written to the database.
 
 ## Other changes