Using Morphia and mapped models to convert BSON to POJO

[antoni-gierczak]

hi folks,

I'm working on some integration testing I'm aiming to keep constant through from v1 to v2 upgrades.

For this we defined some test data in .bson form. We want to be able to map this to our Morphia models so we can validate some logic. In v1 this is simple by using morphia#fromDBObject once the classes are mapped onto Morphia.

However, I can see from this issue and discussion that it's not included functionality anymore.

I'm wondering if there's anything possible through a v2.3+ upgrade as these methods have been removed from public APIs.

I see the workaround in that discussion is to use some internal classes that use the CodecRegistry. However, we don't have models defined in our codec registry, just specific types. We only map our models with getMapper().map(...), which I can see produces accessible EntityModels. However, I don't see any way a Document can be passed through any kind of Encoder to produce a model that's been mapped. Am I missing something or is this not possible with Morphia v2.3+?

[evanchooly]

OK. So this is going to deep in to the weeds of internal stuff that should get you where you're going but I wouldn't recommend relying on it because internals can always change without warning yadda yadda yadda. But enough with the safe harbor disclaimers...

So to start off with, you're going to have a file with BSON bytes, yeah? First off, you'll want to read that in to memory. Read each bson in to a byte[] and then you can read that in to a BsonDocument:

var bsonDocument = new BsonDocumentCodec()
   .decode(new BsonBinaryReader(ByteBuffer.wrap(bytes)
   .order(ByteOrder.LITTLE_ENDIAN)), DecoderContext.builder().build());

Once that's done, you'll have a BsonDocument and you can create a Document with new Document(bsonDocument). Given that Document reference, we finally get to Morphia.

var entity = datastore.getCodecRegistry()
   .get(<your entity type here>)
   .decode(new DocumentReader(document), decoderContext);

That should get you to the entity you need. As you can see it's quite convoluted and relies on a number of internal details from both morphia and the driver. I would suggest a more resilient route. If you have these documents in a database already, you can export them via mongodump (you can use collection and query filters to prune any unwanted data) and then when you want to load them up in to Morphia, you can simply mongorestore them back in to a database and let Morphia load them as it normally would. This would shield you from any API changes on the driver or Morphia front. In fact, Morphia 3.0's API and 5.0 of the driver are making some changes that might hinder the approach above.

Anyway, I hope that gets you (mostly?) over the hump here. If you have any questions, please don't hesitate to ask and we can see what else needs to happen to get you where you want to be.

[antoni-gierczak]

hi @evanchooly, thank you for the response!

I think I may be missing something here. We have Morphia setup in the following manner, using v2.3:

MapperOptions() mapperOptions = MapperOptions.builder().
     ....
    .codecProvider(new SomeSpecificTypesCodecProvider()) // not models, but things like BigDecimal
    .build();
...

Datastore ds = Morphia.createDataStore(name, client, mapperOptions);

ds.getMapper().map(Model.class);

Unfortunately I'm seeing that calling getCodecRegistry for our models is throwing a No Codec Found exception:

datastore.getCodecRegistry()
   .get(Model.class)...

However, these datastores work as expected for type conversion when performing actual database ops.

I can see from doing runtime introspection into the Codec registry there isn't a specified codec for these types. From my understanding of the library map(Class T) is registering EntityModel. What is the relationship between map(...) and the internal codec registry? I am wondering if codecs should be returned in this instance or if using .map(...) and Codec-level type conversion are exclusive.

[evanchooly]

map() builds out the EntityModel and registers a MorphiaCodec instance for that model. Are you calling map() on your types first?

[antoni-gierczak]

hi @evanchooly , thanks for your response. It turns out this actually was the solution. The issue was at runtime it was failing to fetch a codec for child models (that needed @Entity in v2) but .map(...) was not throwing any issues. It looks like from my exploration creature the MorphiaCodec was happening in some lazy load fashion internally from EntityModels which are built when .map(...) is called. We fixed our models and then this worked exactly as expected.