Record mongoDb problems encountered, and restTemplate request retries

When using mongDb recently, there are some problems.

mongoDb entity id auto increment

I got this error when saving the entity

Cannot aotogenerate id of type java.lang.Long for entity
That is, the id cannot be automatically generated

Then I went to Google with the error. Got the following results:

Mongo ObjectIds do not match jave Long type.

Let's first introduce the object Id (ObjectId) of MongoDB

ObjectId

ObjectId is a 12-byte BSON type data with the following format:

The first 4 bytes represent the timestamp The next 3 bytes are the machine identification code The next two bytes consist of the process id (PID)
The last three bytes are random numbers.

For example 62ee651f cabe7f 59dc f10372

Starting from MongoDB 3.4 (first released in December 2016), the design of ObjectId has been modified, and the value of the middle 5 bytes has been changed from the original "machine identification code + process number" to a purely random value.

a 4-byte value representing the seconds since the Unix epoch,
a 5-byte random value, and
a 3-byte counter, starting with a random value.

---

Documents stored in MongoDB must have an "_id" key. The value of this key can be of any type and defaults to an ObjectId object.

In a collection, each document has a unique "_id" value to ensure that each document in the collection can be uniquely identified.

The main reason MongoDB uses ObjectId over other more conventional practices (such as auto-incrementing primary keys) is that synchronizing auto-incrementing primary key values across multiple servers is laborious and time-consuming.

find out the reason

Later, in the entity definition, I found that the id I defined was of type Long.
Since mongoDb automatically increments the id, the generated id cannot be converted to Long, so an error is reported .

 @JsonIgnoreProperties(ignoreUnknown = true)
@Document
public class System {

  @Id
  @JsonView
  private Long id;
}

After changing to String type, the error disappears.

Check the data in the database and see the ObjectId type .

Passed to the foreground, it is string type data .

Summarize

If you want to set the id automatically in mongoDb, you need to set the @Id annotation and set it to string type.

Choose to use the String property as the ID, and Mongo will automatically assign it a value on save (if it's empty). By choosing String, you can get an automatically assigned ID that is managed by the database without having to bother manually setting the property.

But in the current project, the id is the data obtained from gitlab, so there is no need to set the id automatically.

MongoDB persistent entity mapping

I encountered a problem when I checked the relevant entity annotations on the Internet.

spring-data-mongodb mainly has the following common annotations.

@Id
Primary key, non-repeatable, with its own index

@Document
Annotated on the entity class, similar to the entity annotation of hibernate, indicating that the table is maintained by mongo

@Document(collection="mongodb corresponds to the collection name")

 @Document
public class System {

  @Id
  @JsonView
  private String id;

The difference from before is that instead of @Entity annotation, @Document annotation is used.

@Field : property

Field name to store into the database.

Adding this annotation is to use the value of the annotation to correspond to the key of mongo
Without this annotation, the default parameter corresponds to the mongo key

 @Field("user_name")
private String userName;

In addition, the @CompoundIndex joint index and @Transient: properties will not be described in detail.

---

associated entity

A more important point is the associated entity.

In JPA, we can annotate with @ManyToMany, @OneToMany, etc. to generate intermediate tables and other operations.

But in mongo, these annotations don't seem to work.

After I try to add @ManyToMany, the intermediate table is not generated

@DBRef Annotation

In the mongo entity, annotations similar to @OneToMany are: @DBRef

This annotation indicates that it is associated with another document object. Similar to mysql table associations.

This is the case without this comment . Store it in labels as normal data.

---

This is the case when annotated.
Here are the test results of other articles, I haven't tested them yet

It can be found that only its id and namespace are saved. It is equivalent to a reference, or a foreign key.

It is also worth noting that,
This annotation does not handle cascading saves , you must save the object associated with @DBRef first.

You need to respority.save(the object associated with @DBRef) first, and then save the original object.

Here are the test results from other articles:

summary:
It looks like mongo handles it a little differently. However, although mongo is currently used in the project, annotations such as @Entity and @ManyToMany are still used, and no errors are reported.

But from the result, @ManyToMany did not generate the intermediate table. Not useful for mongo.

restTemplate request retry

When requesting data from gitlab in the background, this error will be reported no matter how many times you try. But others can ask.

SSL was abnormally closed.

This error has also been reported before, and the solutions given on the Internet are generally as follows. Set the https protocol type.

 System.setProperty("https.protocols", "TLSv1,TLSv1.1,TLSv1.2,SSLv3");

But tried several times to no avail.

---

After viewing the code, you need to get the comments under 35 issues. Guess it may be because the number of requests is related to the amount of data.

So I tried to add a retry mechanism when the restTemplate request fails .

1. Installation configuration

Edit the pom.xml file and add related dependencies

 <!-- 重试机制 -->
<dependency>
    <groupId>org.springframework.retry</groupId>
    <artifactId>spring-retry</artifactId>
    <version>1.1.2.RELEASE</version>
</dependency>
<dependency>
    <groupId>org.aspectj</groupId>
    <artifactId>aspectjweaver</artifactId>
</dependency>

2. Add @EnableRetry annotation

 @SpringBootApplication(exclude={DataSourceAutoConfiguration.class})
@EnableRetry
@EnableAsync
public class NonlycodeApplication {

3. Annotate with @Retryable

 @Override
 @Retryable(value = {RestClientException.class, EOFException.class}, maxAttempts = 3,
          backoff = @Backoff(delay = 1000L,multiplier = 1))
  public <T> List<T> httGetWithPageRequest(String url, Map<String, String> variables, Integer latestPage, Class<T[]> valueType) throws JsonProcessingException {

The method annotated with @Retryable will be retried when an exception occurs. Parameter description:
value: specifies a retry exception
maxAttemps: maximum number of retries, default 3
backoff: retry waiting strategy, default no

The @Backoff annotation is the retry waiting strategy, parameter description:
delay: Specify the delay time for retry, the default is 1000L, 1 second
multiplier: specifies the multiple of the delay, the default is 0. For example, when delay=5000l and multiplier=2, the first retry is 5 seconds, the second is 10 seconds, and the third is 20 seconds.

This allows the annotated method to retry whenever one of the exceptions occurs when calling this method.

Since Retryable uses aspect enhancement, there will be a pit in the proxy class, that is, the internal call of the method, which will invalidate the annotation.

Throw an exception

An exception is thrown when the number of retries is exceeded.

 //发起请求
    RestTemplate restTemplate = new RestTemplate(new HttpsClientRequestFactory());
    ResponseEntity<String> response = restTemplate.exchange(requestUrl, HttpMethod.GET, new HttpEntity<String>(headers), String.class, variables);

    // 判断请求是否发生异常
    if(!response.getStatusCode().is2xxSuccessful()){
      System.out.println("请求失败...");
      // 抛出异常
      throw new RestClientException(response.getBody());
    }

We can define a global exception handling class. Receive the thrown exception and return it to the foreground.

 @ControllerAdvice
public class CustomExceptionHandler {
    @ExceptionHandler(RestClientException.class)
    public ResponseEntity<String> throwRestException(RestClientException restClientException){
        return new ResponseEntity<String>(restClientException.getMessage(),
                HttpStatus.BAD_REQUEST);
    }
}

After testing, it works fine and all requests are successfully completed.