别拿豆包不当干粮

别拿豆包不当干粮 查看完整档案

杭州编辑  |  填写毕业院校  |  填写所在公司/组织 null.renhy.top 编辑
编辑
_ | |__ _ _ __ _ | '_ \| | | |/ _` | | |_) | |_| | (_| | |_.__/ \__,_|\__, | |___/ 该用户太懒什么也没留下

个人动态

别拿豆包不当干粮 发布了文章 · 2019-09-04

springboot+mybatis实现数据库的读写分离

介绍

随着业务的发展,除了拆分业务模块外,数据库的读写分离也是常见的优化手段。
方案使用了AbstractRoutingDataSourcemybatis plugin来动态的选择数据源
选择这个方案的原因主要是不需要改动原有业务代码,非常友好

注:
demo中使用了mybatis-plus,实际使用mybatis也是一样的
demo中使用的数据库是postgres,实际任一类型主从备份的数据库示例都是一样的
demo中使用了alibaba的druid数据源,实际其他类型的数据源也是一样的

环境

首先,我们需要两个数据库实例,一为master,一为slave。
所有的写操作,我们在master节点上操作
所有的读操作,我们在slave节点上操作

需要注意的是:对于一次有读有写的事务,事务内的读操作也不应该在slave节点上,所有操作都应该在master节点上

先跑起来两个pg的实例,其中15432端口对应的master节点,15433端口对应的slave节点:

docker run \
    --name pg-master \
    -p 15432:5432 \
    --env 'PG_PASSWORD=postgres' \
    --env 'REPLICATION_MODE=master' \
    --env 'REPLICATION_USER=repluser' \
       --env 'REPLICATION_PASS=repluserpass' \
    -d sameersbn/postgresql:10-2

docker run \
    --name pg-slave \
    -p 15433:5432 \
    --link pg-master:master \
    --env 'PG_PASSWORD=postgres' \
    --env 'REPLICATION_MODE=slave' \
    --env 'REPLICATION_SSLMODE=prefer' \
    --env 'REPLICATION_HOST=master' \
    --env 'REPLICATION_PORT=5432' \
    --env 'REPLICATION_USER=repluser' \
       --env 'REPLICATION_PASS=repluserpass' \
    -d sameersbn/postgresql:10-2

实现

整个实现主要有3个部分:

  • 配置两个数据源
  • 实现AbstractRoutingDataSource来动态的使用数据源
  • 实现mybatis plugin来动态的选择数据源

配置数据源

将数据库连接信息配置到application.yml文件中

spring:
  mvc:
    servlet:
      path: /api

datasource:
  write:
    driver-class-name: org.postgresql.Driver
    url: "${DB_URL_WRITE:jdbc:postgresql://localhost:15432/postgres}"
    username: "${DB_USERNAME_WRITE:postgres}"
    password: "${DB_PASSWORD_WRITE:postgres}"
  read:
    driver-class-name: org.postgresql.Driver
    url: "${DB_URL_READ:jdbc:postgresql://localhost:15433/postgres}"
    username: "${DB_USERNAME_READ:postgres}"
    password: "${DB_PASSWORD_READ:postgres}"


mybatis-plus:
  configuration:
    map-underscore-to-camel-case: true

write写数据源,对应到master节点的15432端口
read读数据源,对应到slave节点的15433端口

将两个数据源信息注入为DataSourceProperties

@Configuration
public class DataSourcePropertiesConfig {

    @Primary
    @Bean("writeDataSourceProperties")
    @ConfigurationProperties("datasource.write")
    public DataSourceProperties writeDataSourceProperties() {
        return new DataSourceProperties();
    }

    @Bean("readDataSourceProperties")
    @ConfigurationProperties("datasource.read")
    public DataSourceProperties readDataSourceProperties() {
        return new DataSourceProperties();
    }
}

实现AbstractRoutingDataSource

spring提供了AbstractRoutingDataSource,提供了动态选择数据源的功能,替换原有的单一数据源后,即可实现读写分离:

@Component
public class CustomRoutingDataSource extends AbstractRoutingDataSource {

    @Resource(name = "writeDataSourceProperties")
    private DataSourceProperties writeProperties;

    @Resource(name = "readDataSourceProperties")
    private DataSourceProperties readProperties;


    @Override
    public void afterPropertiesSet() {
        DataSource writeDataSource = 
            writeProperties.initializeDataSourceBuilder().type(DruidDataSource.class).build();
        DataSource readDataSource = 
            readProperties.initializeDataSourceBuilder().type(DruidDataSource.class).build();
        
        setDefaultTargetDataSource(writeDataSource);

        Map<Object, Object> dataSourceMap = new HashMap<>();
        dataSourceMap.put(WRITE_DATASOURCE, writeDataSource);
        dataSourceMap.put(READ_DATASOURCE, readDataSource);
        setTargetDataSources(dataSourceMap);

        super.afterPropertiesSet();
    }

    @Override
    protected Object determineCurrentLookupKey() {
        String key = DataSourceHolder.getDataSource();

        if (key == null) {
             // default datasource
            return WRITE_DATASOURCE;
        }

        return key;
    }

}

AbstractRoutingDataSource内部维护了一个Map<Object, Object>的Map
在初始化过程中,我们将write、read两个数据源加入到这个map
调用数据源时:determineCurrentLookupKey()方法返回了需要使用的数据源对应的key

当前线程需要使用的数据源对应的key,是在DataSourceHolder类中维护的:

public class DataSourceHolder {

    public static final String WRITE_DATASOURCE = "write";
    public static final String READ_DATASOURCE = "read";

    private static final ThreadLocal<String> local = new ThreadLocal<>();


    public static void putDataSource(String dataSource) {
        local.set(dataSource);
    }

    public static String getDataSource() {
        return local.get();
    }

    public static void clearDataSource() {
        local.remove();
    }

}

实现mybatis plugin

上面提到了当前线程使用的数据源对应的key,这个key需要在mybatis plugin根据sql类型来确定
MybatisDataSourceInterceptor类:

@Component
@Intercepts({
        @Signature(type = Executor.class, method = "update",
                args = {MappedStatement.class, Object.class}),
        @Signature(type = Executor.class, method = "query",
                args = {MappedStatement.class, Object.class, RowBounds.class, ResultHandler.class}),
        @Signature(type = Executor.class, method = "query",
                args = {MappedStatement.class, Object.class, RowBounds.class, ResultHandler.class,
                        CacheKey.class, BoundSql.class})})
public class MybatisDataSourceInterceptor implements Interceptor {

    @Override
    public Object intercept(Invocation invocation) throws Throwable {

        boolean synchronizationActive = TransactionSynchronizationManager.isSynchronizationActive();
        if(!synchronizationActive) {
            Object[] objects = invocation.getArgs();
            MappedStatement ms = (MappedStatement) objects[0];

            if (ms.getSqlCommandType().equals(SqlCommandType.SELECT)) {
                DataSourceHolder.putDataSource(DataSourceHolder.READ_DATASOURCE);
            }
        }

        return invocation.proceed();
    }

    @Override
    public Object plugin(Object target) {
        return Plugin.wrap(target, this);
    }

    @Override
    public void setProperties(Properties properties) {
    }
}

仅当未在事务中,并且调用的sql是select类型时,在DataSourceHolder中将数据源设为read
其他情况下,AbstractRoutingDataSource会使用默认的write数据源

至此,项目已经可以自动的在读、写数据源间切换,无需修改原有的业务代码
最后,提供demo使用依赖版本

<parent>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-parent</artifactId>
    <version>2.1.7.RELEASE</version>
    <relativePath/> <!-- lookup parent from repository -->
</parent>

<dependencies>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter</artifactId>
    </dependency>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-web</artifactId>
    </dependency>

    <dependency>
        <groupId>org.postgresql</groupId>
        <artifactId>postgresql</artifactId>
        <version>42.2.2</version>
    </dependency>
    <dependency>
        <groupId>com.alibaba</groupId>
        <artifactId>druid-spring-boot-starter</artifactId>
        <version>1.1.9</version>
    </dependency>
    <dependency>
        <groupId>com.baomidou</groupId>
        <artifactId>mybatisplus-spring-boot-starter</artifactId>
        <version>1.0.5</version>
    </dependency>
    <dependency>
        <groupId>com.baomidou</groupId>
        <artifactId>mybatis-plus</artifactId>
        <version>2.1.9</version>
    </dependency>
    <dependency>
        <groupId>io.springfox</groupId>
        <artifactId>springfox-swagger2</artifactId>
        <version>2.8.0</version>
    </dependency>
    <dependency>
        <groupId>io.springfox</groupId>
        <artifactId>springfox-swagger-ui</artifactId>
        <version>2.8.0</version>
    </dependency>
    <dependency>
        <groupId>org.projectlombok</groupId>
        <artifactId>lombok</artifactId>
        <version>1.16.20</version>
    </dependency>

    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-test</artifactId>
        <scope>test</scope>
    </dependency>
</dependencies>
查看原文

赞 4 收藏 2 评论 1

别拿豆包不当干粮 发布了文章 · 2019-08-29

敏感词过滤的简易实现

介绍

很多时候我们需要对接受的文本进行过滤,剔除一下不当用词,比如一些反动的、侮辱性的、淫秽的用语
一般会有一个敏感词词库,基于这个词库对输入的文本进行过滤,分享一种简易的实现

示例中为了和谐,将不会出现上述违反社会主义核心价值观的词汇,使用“小明”、“小红”来举例    
实际生产中可以用需要过滤的敏感词列表替换

现在假设“小明”、“小红”已经属于敏感词了,那么理想的效果:

输入:  小明上课吃零食,老师让小红出去。   
输出:  **上课吃零食,老师让**出去。

很多时候用户会简单的插入一些' '、'_'、'%'类似的简单字符来躲避过滤,希望算法同样能过滤敏感词:

输入:  小 明上课吃零食,老师让'小'红'出去
输出:  * *上课吃零食,老师让'*'*'出去

准备

首先,需要准备一个敏感词列表

比如:

  1. 小明
  2. 小红

然后,需要一个高效的匹配算法

因为在实际生产中,敏感词数量会比较多,传入的文本也会比较长
单纯的遍历敏感词列表对字符串使用String.replace(key, "**")效果会比较差
这里使用了一种Aho Corasick自动机结合DoubleArrayTrie极速多模式匹配的算法来进行敏感词的匹配

实现

定义敏感词列表

private static final String[] SENSITIVE_KEYS = new String[]{
    "小明",
    "小红"
};

使用maven将算法库引用进来

<dependencies>
    <dependency>
        <groupId>com.hankcs</groupId>
        <artifactId>aho-corasick-double-array-trie</artifactId>
        <version>1.2.1</version>
    </dependency>
</dependencies>

使用匹配器来匹配敏感词位置并替换为'*'

public static String shadowSensitive(String text) {
    StringBuffer sb = new StringBuffer(text);
    // filter sensitive words
    List<AhoCorasickDoubleArrayTrie.Hit<String>> hits = acdat.parseText(sb);
    // shadow sensitive words
    for (AhoCorasickDoubleArrayTrie.Hit<String> hit : hits) {
        for (int i = hit.begin; i < hit.end; i++ ){
            sb.deleteCharAt(i);
            sb.insert(i, "*");
        }
    }
    return sb.toString();
}

接下来测试一下,需要先初始化一下匹配器

public static void main(String[] args) {
    TreeMap<String, String> keys = new TreeMap<String, String>();
    for (String key : SENSITIVE_KEYS) {
        keys.put(key, key);
    }
    acdat = new AhoCorasickDoubleArrayTrie<String>();
    acdat.build(keys);
    
    String text1 = "小明上课吃零食,老师让小红出去";
    String text2 = "小 明上课吃零食,老师让'小'红'出去";
    
    System.out.println("text1:");
    System.out.println(text1);
    System.out.println(shadowSensitive(text1));
    System.out.println("text2:");
    System.out.println(text2);
    System.out.println(shadowSensitive(text2));
}

执行程序后,控制台输出

text1:
小明上课吃零食,老师让小红出去
**上课吃零食,老师让**出去
text2:
小 明上课吃零食,老师让'小'红'出去
小 明上课吃零食,老师让'小'红'出去

可以看到text1中“小明”和“小红”已经被替换成了“**”
但是,在词语中简单的加入一些字符就可以绕开过滤器,这还需要优化一下

优化

匹配器只能匹配到“小明”,而无法匹配到“小 明”或者“小_明”
优化的思路如下:

  • 输入的文本
小 明上课吃零食,老师让'小'红'出去
  • 将一些常见的字符取出来,只留下文字内容
小明上课吃零食,老师让小红出去
  • 进行敏感词的匹配,将敏感词改为*
**上课吃零食,老师让**出去
  • 将取出的字符再重新插回去
* *上课吃零食,老师让'*'*'出去

参照这个思路,改写了上面的shadowSensitive方法
改线前需要定义一些常见的字符

private static final char[] SPECIAL_CHARS = new char[]{
    ' ', '`', '~', '!', '@', '#', '$', '%', '^', '&', '*', '(', ')', '-', '_', '=','+',
    '\\', '|', '[', '{', ']', '}', ';', ':', '\'', '"', ',', '<', '.', '>', '/','?',
    
    //中文字符
    ' ', '!', '¥', '…', '(', ')', '、', '「', '」', '【', '】', ';', ':', '“', '”', ',', '。', '《', '》', '?'};

为了方便展示,这里仅仅列举了常见的一部分字符,有需要的话,可以随时添加字符进去
优化的shadowSensitive方法:

public static String shadowSensitive(String text) {
    // detect special chars
    List<int[]> descriptors = new ArrayList<int[]>();
    for (int i = 0; i < text.length(); i++) {
        for (int j = 0; j < SPECIAL_CHARS.length; j++) {
            if (text.charAt(i) == SPECIAL_CHARS[j]) {
                int[] descriptor = new int[2];
                descriptor[0] = i;
                descriptor[1] = j;
                descriptors.add(descriptor);
            }
        }
    }
    
    // remove special chars
    StringBuffer sb = new StringBuffer(text);
    for (int i = descriptors.size() - 1; i >= 0; i--) {
        sb.deleteCharAt(descriptors.get(i)[0]);
    }
    
    // filter sensitive words
    List<AhoCorasickDoubleArrayTrie.Hit<String>> hits = acdat.parseText(sb);
    
    // shadow sensitive words
    for (AhoCorasickDoubleArrayTrie.Hit<String> hit : hits) {
        for (int i = hit.begin; i < hit.end; i++ ){
            sb.deleteCharAt(i);
            sb.insert(i, "*");
        }
    }
    
    // refill special chars
    for (int[] descriptor : descriptors) {
        sb.insert(descriptor[0], SPECIAL_CHARS[descriptor[1]]);
    }
    return sb.toString();
}

接下来测试一下

public static void main(String[] args) {
    TreeMap<String, String> keys = new TreeMap<String, String>();
    for (String key : SENSITIVE_KEYS) {
        keys.put(key, key);
    }
    acdat = new AhoCorasickDoubleArrayTrie<String>();
    acdat.build(keys);
    
    String text1 = "小明上课吃零食,老师让小红出去";
    String text2 = "小 明上课吃零食,老师让'小'红'出去";
    
    System.out.println("text1:");
    System.out.println(text1);
    System.out.println(shadowSensitive(text1));
    System.out.println("text2:");
    System.out.println(text2);
    System.out.println(shadowSensitive(text2));
}

执行程序后,控制台输出:

text1:
小明上课吃零食,老师让小红出去
**上课吃零食,老师让**出去
text2:
小 明上课吃零食,老师让'小'红'出去
* *上课吃零食,老师让'*'*'出去

可以看到"小 明"已经修改为""了

查看原文

赞 1 收藏 0 评论 1

别拿豆包不当干粮 发布了文章 · 2019-08-05

SpringBoot中实现子类的反序列化

目标

在SpringBoot接口中,我们一般用@RequestBody类注解需要反序列化的对象,但是当存在多个子类的情况下,常规的反序列化不能满足需求,比如:

我们有一个类Exam用于表示一张试卷:

@Data
public class Exam {

    private String name;
    private List<Question> questions;
}

这里Question比较特殊,Question本身是一个抽象类,提供了一些通用的方法调用,实际子类有单选题、多选题、判断题多种情况
图片描述

实现

SprintBoot内置的序列化是使用的Jackson,查阅文档后发现Jackson提供了@JsonTypeInfo@JsonSubTypes这两个注解,搭配使用,可以根据指定的字段值来指定实例化中用到的具体的子类类型

这几个类的实际代码如下:
抽象基类Question:

@Data
@JsonTypeInfo(
        use = JsonTypeInfo.Id.NAME,
        include = JsonTypeInfo.As.EXISTING_PROPERTY,
        property = "type",
        visible = true)
@JsonSubTypes({
        @JsonSubTypes.Type(value = SingleChoiceQuestion.class, name = Question.SINGLE_CHOICE),
        @JsonSubTypes.Type(value = MultipleChoiceQuestion.class, name = Question.MULTIPLE_CHOICE),
        @JsonSubTypes.Type(value = TrueOrFalseQuestion.class, name = Question.TRUE_OR_FALSE),
})
public abstract class Question {

    protected static final String SINGLE_CHOICE = "single_choice";
    protected static final String MULTIPLE_CHOICE = "multiple_choice";
    protected static final String TRUE_OR_FALSE = "true_or_false";

    protected String type;
    protected String content;
    protected String answer;

    protected boolean isCorrect(String answer) {
        return this.answer.equals(answer);
    }
}

判断题TrueOrFalseQuestion:

@Data
@EqualsAndHashCode(callSuper = true)
public class TrueOrFalseQuestion extends Question {

    public TrueOrFalseQuestion() {
        this.type = TRUE_OR_FALSE;
    }
}

选择题ChoiceQuestion:

@Data
@EqualsAndHashCode(callSuper = true)
public abstract class ChoiceQuestion extends Question {

    private List<Option> options;

    @Data
    public static class Option {
        private String code;
        private String content;
    }
}

单选题SingleChoiceQuestion:

@Data
@EqualsAndHashCode(callSuper = true)
public class SingleChoiceQuestion extends ChoiceQuestion {

    public SingleChoiceQuestion() {
        this.type = SINGLE_CHOICE;
    }
}

多选题MultipleChoiceQuestion:

@Data
@EqualsAndHashCode(callSuper = true)
public class MultipleChoiceQuestion extends ChoiceQuestion {

    public MultipleChoiceQuestion() {
        this.type = MULTIPLE_CHOICE;
    }

    @Override
    public void setAnswer(String answer) {
        this.answer = sortString(answer);
    }

    @Override
    public boolean isCorrect(String answer) {
        return this.answer.equals(sortString(answer));
    }

    private String sortString(String str) {
        char[] chars = str.toCharArray();
        Arrays.sort(chars);
        return String.valueOf(chars);
    }
}

测试

接下来测试一下
定义一个接口,我们可以使用@RequestBody传入一个Exam对象,返回解析结果:

@RequestMapping(value = "/exam", method = RequestMethod.POST)
public List<String> parseExam(@RequestBody Exam exam) {
    List<String> results = new ArrayList<>();
    results.add(String.format("Parsed an exam, name = %s", exam.getName()));
    results.add(String.format("Exam has %s questions", exam.getQuestions().size())) 
    
    List<String> types = new ArrayList<>();
    for (Question question : exam.getQuestions()) {
        types.add(question.getType());
    }
    results.add(String.format("Questions types: %s", types.toString()));
    return results;
}

项目跑起来,调用接口测试一下:

curl -X POST \
  http://127.0.0.1:8080/exam/ \
  -H 'Content-Type: application/json' \
  -d '{
    "name":"一场考试",
    "questions": [
        {
            "type": "single_choice",
            "content": "单选题",
            "options":  [
                {
                    "code":"A",
                    "content": "选项A"
                },{
                    "code":"B",
                    "content": "选项B"
                }],
            "answer": "A"
        },{
            "type": "multiple_choice",
            "content": "多选题",
            "options":  [
                {
                    "code":"A",
                    "content": "选项A"
                },{
                    "code":"B",
                    "content": "选项B"
                }],
            "answer": "AB"
        },{
            "type": "true_or_false",
            "content": "判断题",
            "answer": "True"
        }]
}'

接口返回如下:

[
    "Parsed an exam, name = 一场考试",
    "Exam has 3 questions",
    "Questions types: [single_choice, multiple_choice, true_or_false]"
]

这里不同类型的question,type字段都能正确读取,表明反序列化过程中确实是调用了具体子类对应的类来进行实例化的。

查看原文

赞 0 收藏 0 评论 4

别拿豆包不当干粮 发布了文章 · 2019-08-05

JSch-用java实现服务器远程操作

介绍

前段时间接了一个比较特殊的需求,需要做一个用于部署服务的服务。主要是将一个k8s服务集群部署到远端的服务器上,具体服务器的连接信息会通过接口传入。

本来部署是人工来完成的,无非是将一些必须的文件scp到目标服务器上,然后ssh远程登录,执行一些安装的操作,齐活。安装的流程没什么问题,主要是这些步骤需要使用代码来实现,也就是需要一个支持SSH的client库来执行这些操作

最终选用了JSch(Java Secure Channel),官网介绍:

JSch is a pure Java implementation of SSH2.
JSch allows you to connect to an sshd server and use port forwarding, X11 forwarding, file transfer, etc., and you can integrate its functionality into your own Java programs. JSch is licensed under BSD style license.

实现

为了完成部署服务的任务,需要解决几个问题:

  • SSH连接到远端的服务器
  • 在服务器上执行指令
  • 使用scp命令传输文件
  • 编辑服务器上的文件,主要是为了修改一些配置文件

这里介绍下几个主要的工具方法

远程ssh连接

先定义一个Remote类,用于记录服务器登录信息

@Data
public class Remote {

    private String user = "root";
    private String host = "127.0.0.1";
    private int port = 22;
    private String password = "";
    private String identity = "~/.ssh/id_rsa";
    private String passphrase = "";
}

这里填充了一些默认值,平时用的时候方便一些

JSch使用Session来定义一个远程节点:

public static Session getSession(Remote remote) throws JSchException {
    JSch jSch = new JSch();
    if (Files.exists(Paths.get(remote.getIdentity()))) {
        jSch.addIdentity(remote.getIdentity(), remote.getPassphrase());
    }
    Session session = jSch.getSession(remote.getUser(), remote.getHost(),remote.getPort());
    session.setPassword(remote.getPassword());
    session.setConfig("StrictHostKeyChecking", "no");
    return session;
}

测试一下:

public static void main(String[] args) throws Exception {
    Remote remote = new Remote();
    remote.setHost("192.168.124.20");
    remote.setPassword("123456");
    Session session = getSession(remote);
    session.connect(CONNECT_TIMEOUT);
    if (session.isConnected()) {
        System.out.println("Host({}) connected.", remote.getHost);
    }
    session.disconnect();
}

正确的输入了服务器地址和密码后,连接成功。

这里要提一下,JSch会优先使用填入的ssh_key去尝试登录,尝试失败后才会使用password登录,这点和平时使用ssh命令的交互是一致的,好评~

远程指令

接下来就是编写一个通用的方法,用于在Session上执行命令

public static List<String> remoteExecute(Session session, String command) throws JSchException {
    log.debug(">> {}", command);
    List<String> resultLines = new ArrayList<>();
    ChannelExec channel = null;
    try{
        channel = (ChannelExec) session.openChannel("exec");
        channel.setCommand(command);
        InputStream input = channel.getInputStream();
        channel.connect(CONNECT_TIMEOUT);
        try {
            BufferedReader inputReader = new BufferedReader(newInputStreamReader(input));
            String inputLine = null;
            while((inputLine = inputReader.readLine()) != null) {
                log.debug("   {}", inputLine);
                resultLines.add(inputLine);
            }
        } finally {
            if (input != null) {
                try {
                    input.close();
                } catch (Exception e) {
                    log.error("JSch inputStream close error:", e);
                }
            }
        }
    } catch (IOException e) {
        log.error("IOcxecption:", e);
    } finally {
        if (channel != null) {
            try {
                channel.disconnect();
            } catch (Exception e) {
                log.error("JSch channel disconnect error:", e);
            }
        }
    }
    return resultLines;
}

测试一下:

public static void main(String[] args) throws Exception {
    Remote remote = new Remote();
    remote.setHost("192.168.124.20");
    remote.setPassword("123456");
    Session session = getSession(remote);
    session.connect(CONNECT_TIMEOUT);
    if (session.isConnected()) {
        System.out.println("Host({}) connected.", remote.getHost());
    }
    
    remoteExecute(session, "pwd");
    remoteExecute(session, "mkdir /root/jsch-demo");
    remoteExecute(session, "ls /root/jsch-demo");
    remoteExecute(session, "touch /root/jsch-demo/test1; touch /root/jsch-demo/test2");
    remoteExecute(session, "echo 'It a test file.' > /root/jsch-demo/test-file");
    remoteExecute(session, "ls -all /root/jsch-demo");
    remoteExecute(session, "ls -all /root/jsch-demo | grep test");
    remoteExecute(session, "cat /root/jsch-demo/test-file");
    
    session.disconnect();
}

执行后,日志输出如下内容:

Host(192.168.124.20) connected.
>> pwd
   /root
>> mkdir /root/jsch-demo
>> ls /root/jsch-demo
>> touch /root/jsch-demo/test1; touch /root/jsch-demo/test2
>> echo 'It a test file.' > /root/jsch-demo/test-file
>> ls -all /root/jsch-demo
   total 12
   drwxr-xr-x 2 root root 4096 Jul 30 03:05 .
   drwx------ 6 root root 4096 Jul 30 03:05 ..
   -rw-r--r-- 1 root root    0 Jul 30 03:05 test1
   -rw-r--r-- 1 root root    0 Jul 30 03:05 test2
   -rw-r--r-- 1 root root   16 Jul 30 03:05 test-file
>> ls -all /root/jsch-demo | grep test
   -rw-r--r-- 1 root root    0 Jul 30 03:05 test1
   -rw-r--r-- 1 root root    0 Jul 30 03:05 test2
   -rw-r--r-- 1 root root   16 Jul 30 03:05 test-file
>> cat /root/jsch-demo/test-file
   It a test file.

执行结果令人满意,这些常见的命令都成功了
再次好评~

scp操作

scp操作官方给了很详细的示例scpTo+scpFrom,再次好评~
scpTo:

public static long scpTo(String source, Session session, String destination) {
    FileInputStream fileInputStream = null;
    try {
        ChannelExec channel = (ChannelExec) session.openChannel("exec");
        OutputStream out = channel.getOutputStream();
        InputStream in = channel.getInputStream();
        boolean ptimestamp = false;
        String command = "scp";
        if (ptimestamp) {
            command += " -p";
        }
        command += " -t " + destination;
        channel.setCommand(command);
        channel.connect(CONNECT_TIMEOUT);
        if (checkAck(in) != 0) {
            return -1;
        }
        File _lfile = new File(source);
        if (ptimestamp) {
            command = "T " + (_lfile.lastModified() / 1000) + " 0";
            // The access time should be sent here,
            // but it is not accessible with JavaAPI ;-<
            command += (" " + (_lfile.lastModified() / 1000) + " 0\n");
            out.write(command.getBytes());
            out.flush();
            if (checkAck(in) != 0) {
                return -1;
            }
        }
        //send "C0644 filesize filename", where filename should not include '/'
        long fileSize = _lfile.length();
        command = "C0644 " + fileSize + " ";
        if (source.lastIndexOf('/') > 0) {
            command += source.substring(source.lastIndexOf('/') + 1);
        } else {
            command += source;
        }
        command += "\n";
        out.write(command.getBytes());
        out.flush();
        if (checkAck(in) != 0) {
            return -1;
        }
        //send content of file
        fileInputStream = new FileInputStream(source);
        byte[] buf = new byte[1024];
        long sum = 0;
        while (true) {
            int len = fileInputStream.read(buf, 0, buf.length);
            if (len <= 0) {
                break;
            }
            out.write(buf, 0, len);
            sum += len;
        }
        //send '\0'
        buf[0] = 0;
        out.write(buf, 0, 1);
        out.flush();
        if (checkAck(in) != 0) {
            return -1;
        }
        return sum;
    } catch(JSchException e) {
        log.error("scp to catched jsch exception, ", e);
    } catch(IOException e) {
        log.error("scp to catched io exception, ", e);
    } catch(Exception e) {
        log.error("scp to error, ", e);
    } finally {
        if (fileInputStream != null) {
            try {
                fileInputStream.close();
            } catch (Exception e) {
                log.error("File input stream close error, ", e);
            }
        }
    }
    return -1;
}

scpFrom:

public static long scpFrom(Session session, String source, String destination) {
    FileOutputStream fileOutputStream = null;
    try {
        ChannelExec channel = (ChannelExec) session.openChannel("exec");
        channel.setCommand("scp -f " + source);
        OutputStream out = channel.getOutputStream();
        InputStream in = channel.getInputStream();
        channel.connect();
        byte[] buf = new byte[1024];
        //send '\0'
        buf[0] = 0;
        out.write(buf, 0, 1);
        out.flush();
        while(true) {
            if (checkAck(in) != 'C') {
                break;
            }
        }
        //read '644 '
        in.read(buf, 0, 4);
        long fileSize = 0;
        while (true) {
            if (in.read(buf, 0, 1) < 0) {
                break;
            }
            if (buf[0] == ' ') {
                break;
            }
            fileSize = fileSize * 10L + (long)(buf[0] - '0');
        }
        String file = null;
        for (int i = 0; ; i++) {
            in.read(buf, i, 1);
            if (buf[i] == (byte) 0x0a) {
                file = new String(buf, 0, i);
                break;
            }
        }
        // send '\0'
        buf[0] = 0;
        out.write(buf, 0, 1);
        out.flush();
        // read a content of lfile
        if (Files.isDirectory(Paths.get(destination))) {
            fileOutputStream = new FileOutputStream(destination + File.separator +file);
        } else {
            fileOutputStream = new FileOutputStream(destination);
        }
        long sum = 0;
        while (true) {
            int len = in.read(buf, 0 , buf.length);
            if (len <= 0) {
                break;
            }
            sum += len;
            if (len >= fileSize) {
                fileOutputStream.write(buf, 0, (int)fileSize);
                break;
            }
            fileOutputStream.write(buf, 0, len);
            fileSize -= len;
        }
        return sum;
    } catch(JSchException e) {
        log.error("scp to catched jsch exception, ", e);
    } catch(IOException e) {
        log.error("scp to catched io exception, ", e);
    } catch(Exception e) {
        log.error("scp to error, ", e);
    } finally {
        if (fileOutputStream != null) {
            try {
                fileOutputStream.close();
            } catch (Exception e) {
                log.error("File output stream close error, ", e);
            }
        }
    }
    return -1;
}

另外还有一个公用的方法checkAck:

private static int checkAck(InputStream in) throws IOException {
    int b=in.read();
    // b may be 0 for success,
    //          1 for error,
    //          2 for fatal error,
    //          -1
    if(b==0) return b;
    if(b==-1) return b;
    if(b==1 || b==2){
        StringBuffer sb=new StringBuffer();
        int c;
        do {
            c=in.read();
            sb.append((char)c);
        }
        while(c!='\n');
        if(b==1){ // error
            log.debug(sb.toString());
        }
        if(b==2){ // fatal error
            log.debug(sb.toString());
        }
    }
    return b;
}

测试一下:
我们在项目根目录下新建一个文件test.txt

public static void main(String[] args) throws Exception {
    Remote remote = new Remote();
    remote.setHost("192.168.124.20");
    remote.setPassword("123456");
    Session session = getSession(remote);
    session.connect(CONNECT_TIMEOUT);
    if (session.isConnected()) {
        log.debug("Host({}) connected.", remote.getHost());
    }
    
    remoteExecute(session, "ls /root/jsch-demo/");
    scpTo("test.txt", session, "/root/jsch-demo/");
    remoteExecute(session, "ls /root/jsch-demo/");
    remoteExecute(session, "echo ' append text.' >> /root/jsch-demo/test.txt");
    scpFrom(session, "/root/jsch-demo/test.txt", "file-from-remote.txt");
    
    session.disconnect();
}

日志输出如下:
而且可以看到项目目录下出现了一个文件file-from-remote.txt。里面的内容比原先的test.txt多了 append text

Host(192.168.124.20) connected.
>> ls /root/jsch-demo/
   test1
   test2
   test-file
>> ls /root/jsch-demo/
   test1
   test2
   test-file
   test.txt
>> echo ' append text.' >> /root/jsch-demo/test.txt

远程编辑

我们平时在服务器上编辑文件一般使用vi,非常方便,但是在这里操作vi就有点复杂了
最后采用的方案是,先将源文件备份,然后scp拉到本地,编辑完后scp回原位置
remoteEdit方法:

private static boolean remoteEdit(Session session, String source, Function<List<String>, List<String>> process) {
    InputStream in = null;
    OutputStream out = null;
    try {
        String fileName = source;
        int index = source.lastIndexOf('/');
        if (index >= 0) {
            fileName = source.substring(index + 1);
        }
        //backup source
        remoteExecute(session, String.format("cp %s %s", source, source + ".bak." +System.currentTimeMillis()));
        //scp from remote
        String tmpSource = System.getProperty("java.io.tmpdir") + session.getHost() +"-" + fileName;
        scpFrom(session, source, tmpSource);
        in = new FileInputStream(tmpSource);
        //edit file according function process
        String tmpDestination = tmpSource + ".des";
        out = new FileOutputStream(tmpDestination);
        List<String> inputLines = new ArrayList<>();
        BufferedReader reader = new BufferedReader(new InputStreamReader(in));
        String inputLine = null;
        while ((inputLine = reader.readLine()) != null) {
            inputLines.add(inputLine);
        }
        List<String> outputLines = process.apply(inputLines);
        for (String outputLine : outputLines) {
            out.write((outputLine + "\n").getBytes());
            out.flush();
        }
        //scp to remote
        scpTo(tmpDestination, session, source);
        return true;
    } catch (Exception e) {
        log.error("remote edit error, ", e);
        return false;
    } finally {
        if (in != null) {
            try {
                in.close();
            } catch (Exception e) {
                log.error("input stream close error", e);
            }
        }
        if (out != null) {
            try {
                out.close();
            } catch (Exception e) {
                log.error("output stream close error", e);
            }
        }
    }
}

测试一下:

public static void main(String[] args) throws Exception {
    Remote remote = new Remote();
    remote.setHost("192.168.124.20");
    remote.setPassword("123456");
    Session session = getSession(remote);
    session.connect(CONNECT_TIMEOUT);
    if (session.isConnected()) {
        log.debug("Host({}) connected.", remote.getHost());
    }
    
    remoteExecute(session, "echo 'It a test file.' > /root/jsch-demo/test");
    remoteExecute(session, "cat /root/jsch-demo/test");
    remoteEdit(session, "/root/jsch-demo/test", (inputLines) -> {
        List<String> outputLines = new ArrayList<>();
        for (String inputLine : inputLines) {
            outputLines.add(inputLine.toUpperCase());
        }
        return outputLines;
    });
    remoteExecute(session, "cat /root/jsch-demo/test");
    
    session.disconnect();
}

执行后日志输出:

Host(192.168.124.20) connected.
>> echo 'It a test file.' > /root/jsch-demo/test
>> cat /root/jsch-demo/test
   It a test file.
>> cp /root/jsch-demo/test /root/jsch-demo/test.bak.1564556060191
>> cat /root/jsch-demo/test
   IT A TEST FILE.

可以看到字母已经都是大写了

总结

上面这些方法,基本上覆盖了我们日常在服务器上进行操作的场景了,那么不管部署服务,还是运维服务器都不成问题了

查看原文

赞 1 收藏 0 评论 1

别拿豆包不当干粮 关注了专栏 · 2019-05-16

大前端进阶

进阶无所不能

关注 3896

别拿豆包不当干粮 关注了专栏 · 2019-05-16

有赞技术

有赞技术相关内容

关注 6669

别拿豆包不当干粮 关注了专栏 · 2019-05-16

超神经HyperAI

AI 行业实验媒体,站在科技与人文的交叉口,看懂人工智能。微信公众号:HyperAI

关注 4681

别拿豆包不当干粮 关注了用户 · 2019-05-16

智联大前端 @zpfe

您好, 我们是【智联大前端​】。

我们帮助芸芸众生找到更好的工作,当然也不愿错过走在前端之巅的您。

我们在 zpfe@group.zhaopin.com.cn 恭候您的简历。

关注 3793

别拿豆包不当干粮 关注了专栏 · 2019-05-16

前端巅峰

注重前端性能优化和前沿技术,重型跨平台开发,即时通讯技术等。 欢迎关注微信公众号:前端巅峰

关注 16738

别拿豆包不当干粮 关注了专栏 · 2019-05-16

宜信技术学院

宜信技术学院是宜信旗下的金融科技平台。专注分享金融科技深度文章。

关注 11390

认证与成就

  • 获得 7 次点赞
  • 获得 1 枚徽章 获得 0 枚金徽章, 获得 0 枚银徽章, 获得 1 枚铜徽章

擅长技能
编辑

开源项目 & 著作
编辑

(゚∀゚ )
暂时没有

注册于 2019-05-16
个人主页被 172 人浏览