Can't swap the value of a unique property between two objects

[solnic]

Hopefully the ticket title explains the problem well enough, I'm too tired to go into prose...

This came about with integers, as we were trying to re-implement a simple for of is-list. But I've written an example from scratch to illustrate the problem:

require 'rubygems'

require 'dm-core'
require 'dm-validations'
require 'spec'

SQLITE_FILE = File.join(`pwd`.chomp, "test.db")

DataMapper.setup(:default, "sqlite3:#{SQLITE_FILE}")
DataMapper.setup(:reloaded, "sqlite3:#{SQLITE_FILE}")

class Farm
  include DataMapper::Resource

  property :id, Serial

  has n, :cows, :order => [:cow_id]
end

class Cow
  include DataMapper::Resource

  property :id, Serial
  property :cow_id, Integer
  property :name, String

  # Comment out the following line to make the spec pass
  validates_is_unique :name, :scope => [:farm]

  belongs_to :farm
end

module IdentityMapHelper
  def with_db_reconnect(&blk)
    original_repository = DataMapper::Repository.context.pop
    repository(:reloaded, &blk)
    DataMapper::Repository.context << original_repository
  end
end

Spec::Runner.configure do |config|
  include IdentityMapHelper

  config.before(:each) do
    Farm.auto_migrate!
    Cow.auto_migrate!
  end

  config.before(:each) do    
    DataMapper::Repository.context << repository(:default)
  end

  config.after(:each) do
    DataMapper::Repository.context.pop
  end
end

describe "Renaming a cow" do
  before(:each) do
    @bluebell_farm = Farm.create!
    @cow_1 = Cow.create(:cow_id => 1, :name => "Daisy")
    @cow_2 = Cow.create(:cow_id => 1, :name => "Florence")
    @cow_3 = Cow.create(:cow_id => 1, :name => "Tastyburger")

    @bluebell_farm.cows << @cow_1 << @cow_2 << @cow_3
    @bluebell_farm.save
  end

  it "should let you rename the cows" do
    @cow_1.name = "Florence"
    @cow_2.name = "Daisy"
    @bluebell_farm.save

    with_db_reconnect do
      bluebell_farm_reloaded = Farm.get(@bluebell_farm.id)
      cow_1_reloaded = bluebell_farm_reloaded.cows[0]
      cow_2_reloaded = bluebell_farm_reloaded.cows[1]
      cow_1_reloaded.name.should == "Florence"
      cow_2_reloaded.name.should == "Daisy"
    end
  end
end

---

Created by Ashley Moran - 2009-06-30 20:08:34 UTC

Original Lighthouse ticket: http://datamapper.lighthouseapp.com/projects/20609/tickets/936

[solnic]

I wonder if this is even possible without dm-validations having knowledge about what objects are instantiated, so that it knows how to handle uniqueness checking. Outside of a repository block it would be mostly impossible for it to be able to find all initialized objects of a specific type.

by Dan Kubb (dkubb)

[solnic]

It would be impossible outside a repository block. It would probably be hard even with one!

Could you make validations use this process on #save?

• check if the property is unique among in-memory resources
• check if the property is unique among persisted resources not in memory
• save if both conditions are true

One problem I can see is that if this is RDBMS-backed, and the uniqueness is enforced there too, you’d need to have the whole thing run in a transaction, and make the check deferred.

Actually, I think this makes more sense in the context of Repository#save rather than Resource#save.

Just out of curiosity - if this problem was solved, what architectural improvements would come from making DataMapper capable of this kind of deferred/distributed checking?

by Ashley Moran

[solnic]

You can handle that by using special validation context. This is pretty much an edge case, I think it is not worth tweaking DM to address that at least at this point. There are more important things piled up.

May I close this ticket?

by Michael Klishin (antares)

[solnic]

I’m not sure it represents an edge case. To me, it represents the class of situations where a new state of the repository is valid but can’t be determined by inspecting each resource against the repository individually. A similar one might be allowing only three "favourite" items, but the first save might temporarily put 4 in the repository.

Can you explain your context-based workaround though?

by Ashley Moran

[solnic]

And is there a way of marking tickets "deferred" or some such, so they are not marked closed but will not show up as noise in the current list of open, active tickets? It would be a shame to lose this. (And I don’t really want to have to use a separate system to track it later.)

by Ashley Moran

[solnic]

Sure. If you add validations context with all the same validations except for uniqueness, you then can swap unique value in that context. This may help in some cases.

In example used in original post this probably won’t work well, it is true. However, to keep track of all instantiated resources we’d need to override the constructor for DataMapper::Resources, and DM already has cumbersome logic of record instantiation (at least it was the case in 0.9 days): loaded resources are instantiated with .allocate, so constructor did not even get called.

by Michael Klishin (antares)

[solnic]

I put it on hold for now

by Michael Klishin (antares)

[solnic]

The point about new resources is a good one. You don’t want checks running against objects you don’t intend to save. I’ve also noticed the way constructors are not called with loaded resources - in fact I rely on it.

by Ashley Moran

[solnic]

Noticed the "hold" status - we can consider this one low priority for now and maybe I will revisit it when more important work is done.

by Ashley Moran

[solnic]

Normally what I do to "defer" a ticket is change the milestone to the next version after the one we’re working on. Other than reviewing new tickets, I mostly just look at the tickets for the next milestone to avoid getting overwhelmed :)

Just because something is for the next milestone doesn’t mean it’ll be done for that one, just that it will be reviewed and accepted or deferred out longer if it’s low priority or if it’s dependent on architectural changes (like this one.

I usually use hold to indicate that we am waiting on some piece of information from the submitter before proceeding.. like a failing test case if we can’t reproduce it, or a stacktrace or something.

by Dan Kubb (dkubb)

[solnic]

Thanks for explaining this. I’m not aware of the conventions on LightHouse/with DataMapper’s use of LightHouse.

by Ashley Moran

[solnic]

I dunno if the way we’re doing it is optimal, but we’re evolving the approach and it seems to be working relatively well with a few tweaks here and there :)

by Dan Kubb (dkubb)

[solnic]

[project:id#20609 not-tagged:"0.10.0" not-tagged:"0.10.1" milestone:id#51895 bulk edit command]

by Dan Kubb (dkubb)

[solnic]

[bulk edit]

by Dan Kubb (dkubb)

[solnic]

I haven’t seen any activity my antares on this in a while, so I am removing it from the 1.0.0 milestone.

I am also removing antares from the assignee since he hasn’t been around in a while and I don’t want this ticket sitting in limbo too long.. others in the community may want to take it on, but may not if it looks like it’s assigned to someone else.

by Dan Kubb (dkubb)

[solnic]

I imagine more people would be put off by the fact the spec uses Bluebell Farm and a cow called Daisy as example data.

Now what I’d really like to do is write a Unit of Work engine, which might give an opportunity to solve this elegantly...

by Ashley Moran

[solnic]

@ashley: I have a uow branch under my personal fork of dm-core:

http://github.com/dkubb/dm-core/commits/uow

I wasn’t able to finish it in time for DM 1.0, and about ~50 specs currently fail with dm-core. I’m pretty confident it’s the right approach though.

It builds up a DAG of resources, then uses a topological sort to order them so that dependencies can be saved in the correct order. It’s much cleaner than the way we currently handle saving inside DM::Resource. This approach is how SQL Alchemy handles saving an object graph properly.

For now, the UoW is meant to be hidden behind a Resource#save call, but in the future I could see it becoming part of the public API. The goal for DM2 will be to allow persistence of normal ruby objects and I think a nice UoW API will be a big part of this.

I will probably resume work on this after 1.0. Hopefully I can get the issues ironed out in time for 1.1

by Dan Kubb (dkubb)

[solnic]

Had no idea you were working on that! I’m excited :-) I have some stuff to work through first but I will hopefully look over the code at some point.

by Ashley Moran

[solnic]

Although it’s DBMS-specific, the "Deferrable unique constraints, now permit mass updates to unique keys" feature of PostgreSQL 9 may be of use here:

http://developer.postgresql.org/pgdocs/postgres/release-9-0.html

Don’t take my word for it, though - I didn’t think about it too hard :)

by Ashley Moran