1.引言

需要从tushare获取股票数据，保存在本地,进行分析.

本文以日线接口为例，记录使用java通过http方式处理请求和响应的方法。

日线接口

https://tushare.pro/document/2?doc_id=27

HTTP接口调用方式

https://tushare.pro/document/1?doc_id=130

为了简化多个接口的访问,实现一些基础类，包括：

.对请求,响应的封装

.对响应的解析,结果业务对象的生成

2.类定义

类包括：

• Request: 表示tushare的http请求体

• Response: 表示tushare的http响应消息

• 对应数据库的Entity类: DailyPrice(日线数据)

2.1Request

/// tushare请求体
@Data
public class Request {
    @JsonProperty("api_name")
    private String apiName;
    private String token;
    private Map<String,String> params = new HashMap<>();
    private List<String> fields = new ArrayList<>();

    public void addParam(String name,String value) {
        params.put(name,value);
    }
}

2.2 Response

@Data
public class Response {
    private String msg;
    private Integer code;


    @lombok.Data
    public class Data {
        private List<String> fields;
        private List<List<String>> items;
    }
    private Data data;
}

3.实现

把响应消息的字段映射到实体属性上.

Response的data,其中,fields包含了返回的字段列表,items是返回的多个记录，按字段列表顺序排列.

Response.Data增加访问字段的方法：

    ///< field名称对应的顺序号,从0开始
    private Map<String,Integer> fieldIndex = new HashMap<>();
    public void indexField() {
        int index = 0;
        for (String field : fields) {
            fieldIndex.put(field,index++);
        }
    }
    ///< 根据列名称获取在fields中的顺序号
    public int getFieldIndex(String name) {
        return Optional.ofNullable(fieldIndex.get(name)).orElse(-1);
    }

    ///< 获取记录指定字段的值
    public String getFieldValue(List<String> record,String name) {
        return record.get(getFieldIndex(name));
    }

这里有2种映射方式:

1.定义映射表

2.利用自定义注解在Entity类的属性上定义

映射可能存在转换,如取字串.

第2种是最终采用的方式,代码更少,易于维护. 实现时利用SpEL支持复杂映射.

3.1基于映射表

定义响应消息字段与目标实体属性的映射关系.

利用反射机制动态构造对象,并设置属性值。

Response.Data增加处理记录的方法

    ////< 把items的一条记录转换为clazz指定类型的对象
    /// @param fieldMap field与对象成员名称的映射关系
    public <T> T handleRecord(List<String> record,Map<String,String> fieldMap,Class<?> clazz) throws Exception {
        Constructor<?> c = clazz.getDeclaredConstructor(new Class[]{});
        T target = (T) c.newInstance();
        for (Map.Entry<String,String> entry : fieldMap.entrySet()) {
            int fieldIndex = getFieldIndex(entry.getValue());
            String value = record.get(fieldIndex);
            Util.setPropertyValue(target,clazz,entry.getKey(),value);
        }
        return target;
    }

使用示例:

private  void readDaily_1() throws Exception {
    Request request = new Request();
    request.setApiName("daily");
    request.setToken(token);
    request.addParam("ts_code","002340.SZ");
    request.addParam("start_date","20190101");
    request.addParam("end_date","20191120");

    Response response = restTemplate.postForObject(url,request,Response.class);
    ///< 属性与响应字段的对应关系:
    ///< 映射定义不完整.由于ts_code与code之间不是简单对应,需要转换.通过程序代码实现转换.
    Map<String,String> fieldMap = Util.toMap(new String[][]{
            {"day","trade_date"},
            {"open","open"},
            {"highest","high"},
            {"lowest","low"},
            {"close","close"},
            {"volume","vol"},
            {"amount","amount"}
    });

    response.getData().indexField();
    DailyPriceRepository dailyPriceRepository = Application.getBean(DailyPriceRepository.class);
    for (List<String> item : response.getData().getItems()) {
        DailyPrice dailyPrice =   response.getData().handleRecord(item,fieldMap, DailyPrice.class,null);
        ///< 返回ts_code的格式是:002340.SZ
        dailyPrice.setCode(response.getData().getFieldValue(item,"ts_code").substring(0,6));

        dailyPriceRepository.save(dailyPrice);
    }
}

3.2注解实现

。定义TushareField注解 用于修饰属性

@Retention(RetentionPolicy.RUNTIME)
@Target(ElementType.FIELD)
public @interface TushareField {
    String value() default "";
    String format() default "";
}

value:字段对应的SpEL表达式.

format:表示Date的格式

。使用TushareField修饰DailyPrice的属性：

@Data
@Table(name = "daily_price")
@IdClass(DailyPrice.DailyPricePK.class)
@Entity
public class DailyPrice implements Serializable {
 private static final long serialVersionUID = 1L;

  @Id
  @Column(insertable = false, name = "code", nullable = false)
  @TushareField(value="new String(#ts_code).substring(0,6)")
  private String code;

  @Id
  @Column(name = "day", insertable = false, nullable = false)
  @TushareField(value="{#trade_date}",format = "yyyyMMdd")
  private Date day;

  @Column(name = "open")
  @TushareField(value="{#open}")
  private BigDecimal open;

  @Column(name = "close")
  @TushareField(value="{#close}")
  private BigDecimal close;

  @Column(name = "highest")
  @TushareField(value="{#high}")
  private BigDecimal highest;

  @Column(name = "lowest")
  @TushareField(value="{#low}")
  private BigDecimal lowest;

  @Column(name = "volume")
  @TushareField(value="{#vol}")
  private BigDecimal volume;

  @Column(name = "amount")
  @TushareField(value="{#amount}")
  private BigDecimal amount;

  @Data
  public static class DailyPricePK implements  Serializable {
    @Column(name="code")
    private String code;

    @Column(name = "day")
    private Date day;
  }
}

code是ts_code的前6个字符.

trade_date的格式为yyyyMMdd.

。处理注解 Response.Data中增加处理注解的代码.

   private  String getTushareFieldExpr(Field field) {
        String annotationValue = field.getAnnotation(TushareField.class).value();
        return annotationValue.isEmpty() ? field.getName() : annotationValue;
    }

    ///< 代码来源:
    /// https://stackoverflow.com/questions/11828778/list-free-variables-in-an-el-expression
    private List<String> getVars(SpelNode node) {
        List<String> vars = new ArrayList<>();
        for (int i = 0; i < node.getChildCount(); i++) {
            SpelNode child = node.getChild(i);
            if (child.getChildCount() > 0) {
                vars.addAll(getVars(child));
            }
            else {
                if (child instanceof VariableReference) {
                    vars.add(child.toStringAST());
                }
            }
        }
        return vars;
    }

    public <T> T handleRecord(List<String> record,Class<?> clazz) throws Exception {
        Constructor<?> c = clazz.getDeclaredConstructor(new Class[]{});
        T target = (T) c.newInstance();

        for (Field field : clazz.getDeclaredFields()) {
            field.setAccessible(true);
            if (field.isAnnotationPresent(TushareField.class)) {
                String fieldExpr = getTushareFieldExpr(field); 
                ExpressionParser parser = new SpelExpressionParser();
                Expression exp = parser.parseExpression(fieldExpr);

                StandardEvaluationContext context = new StandardEvaluationContext();
                SpelNode node = ((SpelExpression)exp).getAST();
                List<String> vars = getVars(node);
                vars.forEach((s)->{
                    String name = s.substring(1);
                    String value = record.get(getFieldIndex(name));
                    context.setVariable(name, value);
                });
                String value = exp.getValue(context,String.class);
                Util.setPropertyValue(target,clazz,field.getName(),value, field.getAnnotation(TushareField.class).format());
            }
        }
        return target;
    }

。使用示例

@Transactional
private  void readDaily() throws Exception {
    /// 构造请求
    Request request = new Request();
    request.setApiName("daily");
    request.setToken(token);
    request.addParam("ts_code","002340.SZ");
    request.addParam("start_date","20190101");
    request.addParam("end_date","20191120");

    ///< 发起请求,获取响应消息
    Response response = restTemplate.postForObject(url,request,Response.class);

    ///< 处理响应,写入数据库
    response.getData().indexField();
    DailyPriceRepository dailyPriceRepository = Application.getBean(DailyPriceRepository.class);
    for (List<String> item : response.getData().getItems()) {
        DailyPrice dailyPrice =   response.getData().handleRecord(item,DailyPrice.class);
        dailyPriceRepository.save(dailyPrice);
    }
}

4.其它代码

@Slf4j
public class Util {
    public static Map<String,String> toMap(String[][] a) {
        Map<String,String> m = new HashMap<>();
        for (String []v : a) {
            m.put(v[0],v[1]);
        }
        return m;
    }

    public static <T> void setPropertyValue(T target,Class<?> c,String name,String value,String format) throws Exception {
        String methodName = "set"+StringUtils.capitalize(name);
        Method[] methods = c.getDeclaredMethods();
        for (Method method :methods) {
            if (!method.getName().equals(methodName))
                continue;
            Class[] para=method.getParameterTypes();
            Class<?> c0 = para[0];
            if (c0.equals(Integer.class))
                method.invoke(target,Integer.valueOf(value));
            else if (c0.equals(String.class))
                method.invoke(target,value);
            else if (c0.equals(Date.class))
                method.invoke(target,toDate(value, Optional.ofNullable(format).orElse("yyyyMMdd")));
            else if (c0.equals(BigDecimal.class))
                method.invoke(target,new BigDecimal(value));
            else
                throw new Exception(String.format("unsupported type:%s",c0.getSimpleName()));
        }
    }

    public static Date toDate(String value,String fmt) throws Exception {
        SimpleDateFormat sdf = new SimpleDateFormat(fmt);
        Date date = sdf.parse(value);
        return date;
    }
}