Custom type doesn't work as the inner type of a map.

[mrcnkoba]

Environment

• Elixir version (elixir -v): Elixir 1.8.1 (compiled with Erlang/OTP 20)
• Database and version (PostgreSQL 9.4, MongoDB 3.2, etc.): PostgreSQL 9.5
• Ecto version (mix deps): 3.1.7
• Database adapter and version (mix deps): postgrex 0.15.0
• Operating system: MacOS 10.14

Current behavior

When I migrated our project from Ecto 2 to 3 I ran into one issue. I can't use a custom Ecto type as the inner_type of a map in {:map, inner_type}. It used to work well with Ecto 2 and I think it might be a regression bug. The custom type that I made stupidly gets rid of NULL bytes from string inputs. I created a small application with tests that reproduces it: https://github.com/mrcnkoba/ecto_3_maybe_bug. In order to reproduce it please:

git clone [email protected]:mrcnkoba/ecto_3_maybe_bug.git
cd ecto_3_maybe_bug
mix deps.get
mix test

My custom type is here: https://github.com/mrcnkoba/ecto_3_maybe_bug/blob/master/lib/ecto3customtypemap/pgstring.ex

As a result I'm getting:

Ecto3customtypemap.Test.BlogPosts
  * test custom type in maps doesn't work (53.3ms)

  1) test custom type in maps doesn't work (Ecto3customtypemap.Test.BlogPosts)
     test/ecto3customtypemap_web/controllers/blog_post_test.exs:6
     ** (Postgrex.Error) ERROR 22P05 (untranslatable_character) unsupported Unicode escape sequence

     \u0000 cannot be converted to text.
     code: Ecto3customtypemap.Repo.insert(c)
     stacktrace:
       (ecto_sql) lib/ecto/adapters/sql.ex:621: Ecto.Adapters.SQL.raise_sql_call_error/1
       (ecto) lib/ecto/repo/schema.ex:649: Ecto.Repo.Schema.apply/4
       (ecto) lib/ecto/repo/schema.ex:262: anonymous fn/15 in Ecto.Repo.Schema.do_insert/4
       test/ecto3customtypemap_web/controllers/blog_post_test.exs:11: (test)

Expected behavior

Record should be inserted into the database without a NULL byte.

[michalmuskala]

Unfortunately this is expected behaviour. Map types are serialized using the JSON encoder directly - they don't go through the regular dump/load cycle other types go through. This is started explicitly for the embedded schemas, but I couldn't find it for maps. We could definitely improve documentation in that area, but changes in behaviour are unlikely to happen here.

I also don't believe this is a regression - the behaviour in this area didn't change between Ecto 2 and Ecto 3.

ecto/lib/ecto/schema.ex

Lines 1473 to 1484 in 854d58e

	## Encoding and decoding
	Because many databases do not support direct encoding and decoding
	of embeds, it is often emulated by Ecto by using specific encoding
	and decoding rules.
	For example, PostgreSQL will store embeds on top of JSONB columns,
	which means types in embedded schemas won't go through the usual
	dump->DB->load cycle but rather encode->DB->decode->cast. This means
	that, when using embedded schemas with databases like PG or MySQL,
	make sure all of your types can be JSON encoded/decoded correctly.
	Ecto provides this guarantee for all built-in types.

[josevalim]

@lucasmazza has also requested this feature and it is also an issue here: danielberkompas/cloak#84

My proposed solution is to introduce a new embed_as(format) :: :dump | :self function in the type behaviour.

[josevalim]

This is now in master.