background
In the Gnss power optimization project, we need to provide an interface to other modules, that is, the power consumption information in our Imp class. However, our power consumption information is deleted after each upload. Therefore, multi-threaded concurrent operation variables occur. The records are as follows:;
Scene reproduction
I simulated the following situations in the project. The codes are as follows:
public static List<Integer> list = new ArrayList();
public static void main(String[] args) {
for (int i = 0; i < 100000; i++) {
list.add(i);
}
System.out.println("list.toString()");
Thread t1 = new Thread(() -> {
while (true) {
System.out.println("AAA----is running");
try {
Thread.sleep(100);
} catch (InterruptedException e) {
e.printStackTrace();
}
//
System.out.println(list.toString() + "\n");
}
});
t1.setName("AAAAA");
t1.start();
Thread t2 = new Thread(() -> {
while (true) {
System.out.println("BBB----is running");
try {
Thread.sleep(50);
list.clear();
list = null;
list = new ArrayList();
for (int i = 0; i < 100000; i++) {
list.add(i);
}
} catch (InterruptedException e) {
e.printStackTrace();
}
}
});
t2.setName("BBBBB");
t2.start();
}
The console outputs are as follows:
list.toString() AAA----is running BBB----is running BBB----is running Exception in thread "AAAAA" java.util.ConcurrentModificationException at java.util.ArrayList$Itr.checkForComodification(ArrayList.java:911) at java.util.ArrayList$Itr.next(ArrayList.java:861) at java.util.AbstractCollection.toString(AbstractCollection.java:461) at com.yifan.funwithwatermark.Main.lambda$main$0(Main.java:33) at java.lang.Thread.run(Thread.java:748) BBB----is running BBB----is running BBB----is running ...
It can be seen that thread A exited abnormally while thread B was running normally
java.util.ConcurrentModificationException
Abnormal generation
When we iterate over an ArrayList or HashMap, if we try to modify the collection (for example, delete elements), we may throw a java util. Exception for concurrentmodificationexception.
Abnormal cause
The parent class of ArrayList, AbstarctList, has a field modCount. Each time you modify the collection (add elements, delete elements...), the field modCount++
The implementation principle behind foreach is actually the Iterator Java Design Pattern: Iterator ), equivalent to the code in the comment section. Here, there is a variable expectedModCount in the Iterator of the iterated ArrayList. This variable will be initialized to be equal to the modCount. However, if the modCount of the collection is modified next, it will cause expectedModCount= modCount, and java util. Concurrentmodificationexception exception
The process is as follows:
11111
Let's go through this process in detail according to the source code
/*
*AbstarctList Inner class of for iteration
*/
private class Itr implements Iterator<E> {
int cursor = 0; //Index of the element to be accessed
int lastRet = -1; //Index of the last access element
int expectedModCount = modCount;//expectedModCount is the expected modification value, and initialization is equal to modCount (a member variable in the AbstractList class)
//Determine if there is another element
public boolean hasNext() {
return cursor != size();
}
//Take out the next element
public E next() {
checkForComodification(); //A key line of code to determine whether expectedModCount and modCount are equal
try {
E next = get(cursor);
lastRet = cursor++;
return next;
} catch (IndexOutOfBoundsException e) {
checkForComodification();
throw new NoSuchElementException();
}
}
public void remove() {
if (lastRet == -1)
throw new IllegalStateException();
checkForComodification();
try {
AbstractList.this.remove(lastRet);
if (lastRet < cursor)
cursor--;
lastRet = -1;
expectedModCount = modCount;
} catch (IndexOutOfBoundsException e) {
throw new ConcurrentModificationException();
}
}
final void checkForComodification() {
if (modCount != expectedModCount)
throw new ConcurrentModificationException();
}
}
According to the code, the three member variables of Itr will be initialized each time the list is iterated
int cursor = 0; //Index of the element to be accessed int lastRet = -1; //Index of the last access element int expectedModCount = modCount; //Expected modified value, initialization equal to modCount (a member variable in the AbstractList class)
Then call hasNext() to loop to determine whether the subscript of the access element has reached the end. If not, call the next() method to fetch the element.
The reason for the exception in the top test code is that modcount is found when the next() method calls checkForComodification()= expectedModCount
Next, let's take a look at the source code of ArrayList to understand how modCount is not equal to expectedModCount.
public boolean add(E paramE) {
ensureCapacityInternal(this.size + 1);
/** Omit code here */
}
private void ensureCapacityInternal(int paramInt) {
if (this.elementData == EMPTY_ELEMENTDATA)
paramInt = Math.max(10, paramInt);
ensureExplicitCapacity(paramInt);
}
private void ensureExplicitCapacity(int paramInt) {
this.modCount += 1; //Modify modCount
/** Omit code here */
}
public boolean remove(Object paramObject) {
int i;
if (paramObject == null)
for (i = 0; i < this.size; ++i) {
if (this.elementData[i] != null)
continue;
fastRemove(i);
return true;
}
else
for (i = 0; i < this.size; ++i) {
if (!(paramObject.equals(this.elementData[i])))
continue;
fastRemove(i);
return true;
}
return false;
}
private void fastRemove(int paramInt) {
this.modCount += 1; //Modify modCount
/** Omit code here */
}
public void clear() {
this.modCount += 1; //Modify modCount
/** Omit code here */
}
From the above code, we can see that the add, remove and clear methods of ArrayList will cause the modCount to change. How to call these methods during the iteration process will cause the increase of modCount, making expectedModCount and modCount in the iteration class unequal.
Exception resolution
1. single thread environment
OK, now we have basically understood the cause of exception sending. Next, let's solve it.
I am self willed. I just want to delete the elements of the collection when iterating over the collection. What should I do?
Iterator<String> iter = list.iterator();
while(iter.hasNext()){
String str = iter.next();
if( str.equals("B") )
{
iter.remove();
}
}
Careful friends will find that there is also a remove method in Itr, which actually calls remove in ArrayList, but adds expectedModCount = modCount; Guaranteed not to throw java util. Concurrentmodificationexception exception.
However, this approach has two drawbacks
1. only remove can be performed. add, clear and other itrs do not.
2. it is only applicable to single thread environment.
2. multithreaded environment
In a multithreaded environment, let's try the above code again
public class Test2 {
static List<String> list = new ArrayList<String>();
public static void main(String[] args) {
list.add("a");
list.add("b");
list.add("c");
list.add("d");
new Thread() {
public void run() {
Iterator<String> iterator = list.iterator();
while (iterator.hasNext()) {
System.out.println(Thread.currentThread().getName() + ":"
+ iterator.next());
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
};
}.start();
new Thread() {
public synchronized void run() {
Iterator<String> iterator = list.iterator();
while (iterator.hasNext()) {
String element = iterator.next();
System.out.println(Thread.currentThread().getName() + ":"
+ element);
if (element.equals("c")) {
iterator.remove();
}
}
};
}.start();
}
}
The reason for the exception is very simple. One thread modifies the modCount of the list, resulting in the difference between the modCount and the expectedModCount of the iterator when another thread iterates.
There are two ways to do this:
- Locking before iteration solves the problem of multithreading, but iteration add, clear and other operations are still unavailable
public class Test2 {
static List<String> list = new ArrayList<String>();
public static void main(String[] args) {
list.add("a");
list.add("b");
list.add("c");
list.add("d");
new Thread() {
public void run() {
Iterator<String> iterator = list.iterator();
synchronized (list) {
while (iterator.hasNext()) {
System.out.println(Thread.currentThread().getName()
+ ":" + iterator.next());
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
};
}.start();
new Thread() {
public synchronized void run() {
Iterator<String> iterator = list.iterator();
synchronized (list) {
while (iterator.hasNext()) {
String element = iterator.next();
System.out.println(Thread.currentThread().getName()
+ ":" + element);
if (element.equals("c")) {
iterator.remove();
}
}
}
};
}.start();
}
}
- CopyOnWriteArrayList is used to solve the problem of multi threading, and add, clear and other operations can be performed at the same time
public class Test2 {
static List<String> list = new CopyOnWriteArrayList<String>();
public static void main(String[] args) {
list.add("a");
list.add("b");
list.add("c");
list.add("d");
new Thread() {
public void run() {
Iterator<String> iterator = list.iterator();
while (iterator.hasNext()) {
System.out.println(Thread.currentThread().getName()
+ ":" + iterator.next());
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
};
}.start();
new Thread() {
public synchronized void run() {
Iterator<String> iterator = list.iterator();
while (iterator.hasNext()) {
String element = iterator.next();
System.out.println(Thread.currentThread().getName()
+ ":" + element);
if (element.equals("c")) {
list.remove(element);
}
}
};
}.start();
}
}
CopyOnWriteArrayList is also a thread safe ArrayList. Its implementation principle is that each add,remove and other operations re create a new array, and then point the reference to the new array.
Since I use less CopyOnWriteArrayList, I won't discuss it here. If you want to know more, you can see: Java Concurrent Programming: CopyOnWriteArrayList of concurrent containers
Deep understanding of exception fail fast mechanism
By now, we seem to have understood the cause of this exception.
However, after careful consideration, there are still some doubts:
- Since modCount and expectedModCount are different and cause exceptions, why set this variable
- ConcurrentModificationException can be translated into "concurrent modification exception". Is this exception related to multithreading?
Let's take a look at the comments of modCount in the source code
/**
* The number of times this list has been <i>structurally modified</i>.
* Structural modifications are those that change the size of the
* list, or otherwise perturb it in such a fashion that iterations in
* progress may yield incorrect results.
*
* <p>This field is used by the iterator and list iterator implementation
* returned by the {@code iterator} and {@code listIterator} methods.
* If the value of this field changes unexpectedly, the iterator (or list
* iterator) will throw a {@code ConcurrentModificationException} in
* response to the {@code next}, {@code remove}, {@code previous},
* {@code set} or {@code add} operations. This provides
* <i>fail-fast</i> behavior, rather than non-deterministic behavior in
* the face of concurrent modification during iteration.
*
* <p><b>Use of this field by subclasses is optional.</b> If a subclass
* wishes to provide fail-fast iterators (and list iterators), then it
* merely has to increment this field in its {@code add(int, E)} and
* {@code remove(int)} methods (and any other methods that it overrides
* that result in structural modifications to the list). A single call to
* {@code add(int, E)} or {@code remove(int)} must add no more than
* one to this field, or the iterators (and list iterators) will throw
* bogus {@code ConcurrentModificationExceptions}. If an implementation
* does not wish to provide fail-fast iterators, this field may be
* ignored.
*/
protected transient int modCount = 0;
We notice that fail fast frequently appears in annotations
So what is the fail fast mechanism?
"Fast fail" is also known as fail fast, which is an error detection mechanism for Java collections. When multiple threads change the structure of A collection, A fail fast mechanism may be generated. Remember that it is possible, not certain. For example, suppose there are two threads (thread 1 and thread 2). Thread 1 traverses the elements in set A through the Iterator, and thread 2 modifies the structure of set A at some time (it is the modification above the structure, rather than simply modifying the contents of the set elements). Then this time program will throw A ConcurrentModificationException exception, resulting in A fail fast mechanism.
After seeing this, we understand that the fail fast mechanism is a mechanism to prevent concurrency problems caused by multithreading modifying collections.
The reason why modCount is a member variable is to identify the errors that occur when multiple threads modify the collection. And java util. Concurrentmodificationexception is a concurrent exception.
However, this exception may also be thrown when a single thread is used.
reference resources: Java improvement (34) -- fail fast mechanism