"Failed to acquire the lock" Exception

[phishman3579]

From VanillaChronicleHash

java.lang.IllegalStateException: java.lang.IllegalStateException: Failed to acquire the lock

I've been seeing this exception occasionally when using chronicle map. From the previous issues (#376) the problem was thought to be fixed. I am currently using version 3.24ea3. We've been experimenting locally and have found that making the lock a class local field seems to have mitigated the exception.
See

Chronicle-Map/src/main/java/net/openhft/chronicle/hash/impl/VanillaChronicleHash.java

Line 93 in 84f4a8e

static final TryAcquireOperation<LockingStrategy> GLOBAL_MUTABLE_STATE_LOCK_TRY_ACQUIRE_OPERATION =

Chronicle-Map/src/main/java/net/openhft/chronicle/hash/impl/VanillaChronicleHash.java

Line 95 in 84f4a8e

static final AcquisitionStrategy<LockingStrategy, RuntimeException> GLOBAL_MUTABLE_STATE_LOCK_ACQUISITION_STRATEGY =

private final LockingStrategy GLOBAL_MUTABLE_STATE_LOCKING_STRATEGY = VanillaReadWriteUpdateWithWaitsLockingStrategy.instance();
private final TryAcquireOperation<LockingStrategy> GLOBAL_MUTABLE_STATE_LOCK_TRY_ACQUIRE_OPERATION = TryAcquireOperations.lock();

Is there a reason why that lock is a static field?

[phishman3579]

We were able to get a lock dump also.

Unable to acquire lock!
Lock value was = -2147483648
BS address:   7f32dda23000d   NTZ: 12
Lock Address: 7f32dda23840d   NTZ: 6
ByteStore:
00000000 2f 7e 53 a0 3f ea c9 a6  fd 07 00 00 21 6e 65 74 /~S????? ????!net
00000010 2e 6f 70 65 6e 68 66 74  2e 63 68 72 6f 6e 69 63 .openhft .chronic
00000020 6c 65 2e 6d 61 70 2e 56  61 6e 69 6c 6c 61 43 68 le.map.V anillaCh
00000030 72 6f 6e 69 63 6c 65 4d  61 70 20 7b 0a 20 20 64 ronicleM ap {?  d
00000040 61 74 61 46 69 6c 65 56  65 72 73 69 6f 6e 3a 20 ataFileV ersion:
00000050 22 33 2e 32 34 65 61 33  22 2c 0a 20 20 6b 65 79 "3.24ea3 ",?  key
00000060 43 6c 61 73 73 3a 20 21  74 79 70 65 20 6c 6f 6e Class: ! type lon
00000070 67 2c 0a 20 20 6b 65 79  53 69 7a 65 4d 61 72 73 g,?  key SizeMars
00000080 68 61 6c 6c 65 72 3a 20  21 6e 65 74 2e 6f 70 65 haller:  !net.ope
00000090 6e 68 66 74 2e 63 68 72  6f 6e 69 63 6c 65 2e 68 nhft.chr onicle.h
000000a0 61 73 68 2e 73 65 72 69  61 6c 69 7a 61 74 69 6f ash.seri alizatio
000000b0 6e 2e 69 6d 70 6c 2e 43  6f 6e 73 74 61 6e 74 53 n.impl.C onstantS
000000c0 69 7a 65 4d 61 72 73 68  61 6c 6c 65 72 20 7b 0a izeMarsh aller {?
000000d0 20 20 20 20 63 6f 6e 73  74 61 6e 74 53 69 7a 65     cons tantSize
000000e0 3a 20 38 0a 20 20 7d 2c  0a 20 20 6b 65 79 52 65 : 8?  }, ?  keyRe
000000f0 61 64 65 72 3a 20 21 6e  65 74 2e 6f 70 65 6e 68 ader: !n et.openh
00000100 66 74 2e 63 68 72 6f 6e  69 63 6c 65 2e 68 61 73 ft.chron icle.has
00000110 68 2e 73 65 72 69 61 6c  69 7a 61 74 69 6f 6e 2e h.serial ization.
00000120 69 6d 70 6c 2e 4c 6f 6e  67 4d 61 72 73 68 61 6c impl.Lon gMarshal
00000130 6c 65 72 20 7b 0a 20 20  7d 2c 0a 20 20 6b 65 79 ler {?   },?  key
00000140 44 61 74 61 41 63 63 65  73 73 3a 20 21 6e 65 74 DataAcce ss: !net
00000150 2e 6f 70 65 6e 68 66 74  2e 63 68 72 6f 6e 69 63 .openhft .chronic
00000160 6c 65 2e 68 61 73 68 2e  73 65 72 69 61 6c 69 7a le.hash. serializ
00000170 61 74 69 6f 6e 2e 69 6d  70 6c 2e 4c 6f 6e 67 44 ation.im pl.LongD
00000180 61 74 61 41 63 63 65 73  73 20 7b 0a 20 20 7d 2c ataAcces s {?  },
00000190 0a 20 20 63 68 65 63 6b  73 75 6d 45 6e 74 72 69 ?  check sumEntri
000001a0 65 73 3a 20 74 72 75 65  2c 0a 20 20 61 63 74 75 es: true ,?  actu
000001b0 61 6c 53 65 67 6d 65 6e  74 73 3a 20 32 35 36 2c alSegmen ts: 256,
000001c0 0a 20 20 68 61 73 68 53  70 6c 69 74 74 69 6e 67 ?  hashS plitting
000001d0 3a 20 21 6e 65 74 2e 6f  70 65 6e 68 66 74 2e 63 : !net.o penhft.c
000001e0 68 72 6f 6e 69 63 6c 65  2e 68 61 73 68 2e 69 6d hronicle .hash.im
000001f0 70 6c 2e 48 61 73 68 53  70 6c 69 74 74 69 6e 67 pl.HashS plitting
00000200 24 46 6f 72 50 6f 77 65  72 4f 66 32 53 65 67 6d $ForPowe rOf2Segm
00000210 65 6e 74 73 20 7b 0a 20  20 20 20 62 69 74 73 3a ents {?     bits:
00000220 20 38 0a 20 20 7d 2c 0a  20 20 63 68 75 6e 6b 53  8?  },?   chunkS
00000230 69 7a 65 3a 20 36 34 2c  0a 20 20 6d 61 78 43 68 ize: 64, ?  maxCh
00000240 75 6e 6b 73 50 65 72 45  6e 74 72 79 3a 20 32 38 unksPerE ntry: 28
00000250 35 31 35 2c 0a 20 20 61  63 74 75 61 6c 43 68 75 515,?  a ctualChu
00000260 6e 6b 73 50 65 72 53 65  67 6d 65 6e 74 54 69 65 nksPerSe gmentTie
00000270 72 3a 20 32 38 35 31 35  2c 0a 20 20 73 65 67 6d r: 28515 ,?  segm
00000280 65 6e 74 48 65 61 64 65  72 53 69 7a 65 3a 20 36 entHeade rSize: 6
00000290 34 2c 0a 20 20 74 69 65  72 48 61 73 68 4c 6f 6f 4,?  tie rHashLoo
000002a0 6b 75 70 56 61 6c 75 65  42 69 74 73 3a 20 31 35 kupValue Bits: 15
000002b0 2c 0a 20 20 74 69 65 72  48 61 73 68 4c 6f 6f 6b ,?  tier HashLook
000002c0 75 70 4b 65 79 42 69 74  73 3a 20 31 37 2c 0a 20 upKeyBit s: 17,?
000002d0 20 74 69 65 72 48 61 73  68 4c 6f 6f 6b 75 70 53  tierHas hLookupS
000002e0 6c 6f 74 53 69 7a 65 3a  20 34 2c 0a 20 20 74 69 lotSize:  4,?  ti
000002f0 65 72 48 61 73 68 4c 6f  6f 6b 75 70 43 61 70 61 erHashLo okupCapa
00000300 63 69 74 79 3a 20 38 31  39 32 2c 0a 20 20 6d 61 city: 81 92,?  ma
00000310 78 45 6e 74 72 69 65 73  50 65 72 48 61 73 68 4c xEntries PerHashL
00000320 6f 6f 6b 75 70 3a 20 36  35 35 33 2c 0a 20 20 74 ookup: 6 553,?  t
00000330 69 65 72 48 61 73 68 4c  6f 6f 6b 75 70 49 6e 6e ierHashL ookupInn
00000340 65 72 53 69 7a 65 3a 20  33 32 37 36 38 2c 0a 20 erSize:  32768,?
00000350 20 74 69 65 72 48 61 73  68 4c 6f 6f 6b 75 70 4f  tierHas hLookupO
00000360 75 74 65 72 53 69 7a 65  3a 20 33 32 37 36 38 2c uterSize : 32768,
00000370 0a 20 20 74 69 65 72 46  72 65 65 4c 69 73 74 49 ?  tierF reeListI
00000380 6e 6e 65 72 53 69 7a 65  3a 20 33 35 36 38 2c 0a nnerSize : 3568,?
00000390 20 20 74 69 65 72 46 72  65 65 4c 69 73 74 4f 75   tierFr eeListOu
000003a0 74 65 72 53 69 7a 65 3a  20 33 35 38 34 2c 0a 20 terSize:  3584,?
000003b0 20 74 69 65 72 45 6e 74  72 79 53 70 61 63 65 49  tierEnt rySpaceI
000003c0 6e 6e 65 72 53 69 7a 65  3a 20 31 38 32 34 39 36 nnerSize : 182496
000003d0 30 2c 0a 20 20 74 69 65  72 45 6e 74 72 79 53 70 0,?  tie rEntrySp
000003e0 61 63 65 49 6e 6e 65 72  4f 66 66 73 65 74 3a 20 aceInner Offset:
000003f0 30 2c 0a 20 20 74 69 65  72 45 6e 74 72 79 53 70 0,?  tie rEntrySp
... truncated
[pos: 0, rlim: 476549120, wlim: 476549120, cap: 476549120 ] /~S?????????!net.openhft.chronicle.map.VanillaChronicleMap {?  dataFileVersion: "3.24ea3",?  keyClass: !type long,?  keySizeMarshaller: !net.openhft.chronicle.hash.serialization.impl.ConstantSizeMarshaller {?    constantSize: 8?  },?  keyReader: !net.openhft.chronicle.hash.serialization.impl.LongMarshaller {?  },?  keyDataAccess: !net.openhft.chronicle.hash.serialization.impl.LongDataAccess {?  },?  checksumEntries: true,?  actualSegments: 256,?  hashSplitting: !net.openhft.chronicle.hash.impl.HashSplitting...
dataFileVersion: "3.24ea3"
keyClass: !type long
keySizeMarshaller: !net.openhft.chronicle.hash.serialization.impl.ConstantSizeMarshaller {
  constantSize: 8
}
keyReader: !net.openhft.chronicle.hash.serialization.impl.LongMarshaller {
}
keyDataAccess: !net.openhft.chronicle.hash.serialization.impl.LongDataAccess {
}
checksumEntries: true
actualSegments: 256
hashSplitting: !net.openhft.chronicle.hash.impl.HashSplitting$ForPowerOf2Segments {
  bits: 8
}
chunkSize: 64
maxChunksPerEntry: 28515
actualChunksPerSegmentTier: 28515
segmentHeaderSize: 64
tierHashLookupValueBits: 15
tierHashLookupKeyBits: 17
tierHashLookupSlotSize: 4
tierHashLookupCapacity: 8192
maxEntriesPerHashLookup: 6553
tierHashLookupInnerSize: 32768
tierHashLookupOuterSize: 32768
tierFreeListInnerSize: 3568
tierFreeListOuterSize: 3584
tierEntrySpaceInnerSize: 1824960
tierEntrySpaceInnerOffset: 0
tierEntrySpaceOuterSize: 1824960
tierSize: 1861440
maxExtraTiers: 256
tierBulkSizeInBytes: 59566080
tierBulkInnerOffsetToTiers: 0
tiersInBulk: 32
log2TiersInBulk: 5

[JerryShea]

@phishman3579 can you share a unit test which reproduces the issue?

We are busy with customer work and so unlikely to get this in the short term (and apologies for such a slow response - this one fell between the cracks). If you would like us to prioritise it please review the support offerings at https://chronicle.software/openhft-support/

[tgd]

Hi @phishman3579 - any luck with a unit test reproducing this?

[phishman3579]

Hello @tgd , sorry we weren't able to reproduce with a unit test; we have only seen it sporadically when our application is consuming at very high rates.

[tgd]

Thanks for getting back to us @phishman3579 - I recommend trying out our latest EA version to see if the issue still persists.