Log an issue where soft deletes fail due to repository layer errors

zero, preface

For the convenience of description, we will abstract and simplify the project.
This is a project that uses Angular in the front end and Spring in the back end. One of the ER diagrams of the project is as follows:

It is not difficult to see that there is a one-to-many relationship between townships and communities.

When managing lower-level areas, foreign keys to higher-level areas are required (eg: a community must have a township to which it belongs)

Since queries in these areas are very frequent, in order to reduce the frequency of SQL, a cache is set in the background.
To avoid deletion of data causing errors in the entire system, soft delete is globally enabled.

1. Problem recurrence

Delete an object (township) on the management page of any entity (such as township management), the deleted data is no longer displayed in the list, and you can also see in the database that the deleted object has deleted=1:

However, when other entities are related to query (when setting up townships in the community), the deleted objects can indeed be found, and they can still be saved...

(If you save the deleted data, it will cause the system to report an error)

A brief summary is:

Due to some problems in the project code, soft delete works normally when querying the list paging, but when a foreign key association is required, soft delete is invalid.

2. Troubleshooting

Exclude cache reasons

First of all, from the issue, it may be caused by the back-end cache (similar problems have occurred before). The backend is set to clear the cache on re-login, so testing is simple.
I tried to refresh the browser, log out and log in again, change browsers, etc., but it did not solve the problem. Now the cause of the cache is basically ruled out.

After further investigation, the internal code of the step-by-step query function using the debug mode in Spring found that the deleted information appeared in the return value.

So far, it can be concluded that it is not the problem of the cache but the problem of the query method.

Check the calling relationship

Since there is a problem with the query, why can the deleted data be distinguished normally in the township list?

With doubts, I found the calling relationship between the front and back ends:

The paging query initiated by the township list finally calls the findAll method

In the community->town selector, querying the town will eventually call the findAll method, but the parameters are different

// findAll没有参数
@Override
public List<Town> findAll() {
  return (List<Town>) this.townRepository.findAll();
}

// page有参数
@Override
public Page<Town> page(String name, Pageable pageable) {
  return this.townRepository.findAll(TownSpecs.containingName(name)), pageable);
}

So the preliminary judgment is that the getAll() method of the warehouse layer TownRepository has omitted to write the functions related to soft deletion. However, there is no information about how soft deletion is implemented in the code we have seen so far, so continue to look for it.

Exploring the implementation of soft deletes

Now that I know what the problem is, I will look for it next. In this project, how is soft deletion implemented, and where is the key code that affects the query of the warehouse layer. I found the soft deleted PR from the historical Pull Request. .

It is found that in this project, all entities inherit the base entity. To enable soft deletion, you need to set the deleted and deleteAt fields in the base entity, as well as the related Setter and Getter methods to indicate the deleted and deleted time:

Then add the @SQLDelete annotation to all inherited classes and replace the delete function with "set deleted=1"
The second line @where(clause = "deleted = false") is used to query only data that has not been soft deleted.

At this point, I thought of a stupid method: add the deleted = 0 condition to all findAlls in the warehouse layer, but the problem is that so many entities will generate a lot of repetitive code, and it will not solve the problem fundamentally, so I give up.

So far, I have searched around and still haven't found the answer: it stands to reason that this can take effect, but why is findAll() abnormal?

I compared the code of the latest local version and found that @where(clause = "deleted = false") has been deleted from the inherited entity

Just when I was wondering, I found a link in the code comment, and I opened it to see that it was the soft-deleted blog written by Mr. Pan before, so I read it again.
Spring boot implements soft delete

This is how I found out:
@Where(clause = "deleted = false") will cause us to get a 500 EntiyNotFound error when doing all or page queries.

The solution is also given in the blog: create a SoftDeleteCrudRepository interface, inherit and override the CrudRepository method inside JPA, and manually add the deleted = false condition to the query method (see the blog for the specific code), which can achieve soft deletion and avoid it. 500 error.

3. Unravel the mystery of BUG

So it can be guessed that the problem must be in the SoftDeleteCrudRepository we wrote ourselves, probably because some methods do not have overrides.

Coming to the soft delete repository of this project, the overloaded methods of findAll are as follows:

  @Override
  public Page<T> findAll(Pageable pageable) {
    return this.findAll(this.andDeleteFalseSpecification(null), pageable);
  }

  @Override
  public Page<T> findAll(@Nullable Specification<T> specification, Pageable pageable) {
    return super.findAll(this.andDeleteFalseSpecification(specification), pageable);
  }

  @Override
  public List<T> findAll(@Nullable Specification<T> specification, Sort sort) {
    return super.findAll(this.andDeleteFalseSpecification(specification), sort);
  }

Let's review how the two cases just now are called:

// findAll没有参数
@Override
public List<Town> findAll() {
  return (List<Town>) this.townRepository.findAll();
}

// page有参数
@Override
public Page<Town> page(String name, Pageable pageable) {
  return this.townRepository.findAll(TownSpecs.containingName(name)), pageable);
}

Well, the case is solved, our soft delete class does not have a findAll method that overrides the empty parameter case, so for the empty parameter, the query condition of deleted=1 is not automatically added.

So just add here:

  @Override
  public List<T> findAll(@Nullable Specification<T> specification) {
    return super.findAll(this.andDeleteFalseSpecification(specification));
  }

The bug in this article can disappear in the findAll() method of all warehouses, and new calling methods will appear in the future, and only need to modify the soft delete class.
So far the problem is finally solved.

References

• Spring boot implements soft deletion: https://segmentfault.com/a/1190000015459505

postscript

In fact, the real writing of this project is more complicated than the reference material:

• SoftDeleteCrudRepository is no longer an interface, but an implementation class
• Other repositories do not directly inherit SoftDeleteCrudRepository, but use factory pattern injection
• I didn't understand the code of the factory mode in the project, and in fact, I didn't understand in the end, how other warehouses call SoftDeleteCrudRepository when there is no extends or implements.

Copyright Notice

The author of this article: Hebei University of Technology Mengyunzhi Development Team - Liu Yuxuan
The newcomer is inexperienced. If you have suggestions, welcome to exchange. If you have mistakes, please feel free to spray.