Protocol Buffer序列化Java框架-Protostuff

了解Protocol Buffer

首先要知道什么是Protocol Buffer，在编程过程中，当涉及数据交换时，我们往往需要将对象进行序列化然后再传输。常见的序列化的格式有JSON，XML等，这些格式虽然可读性较好，但占用的空间大小并不是最优的。基于此，Google创建了一种名叫Protocol Buffer的序列化格式，它与JSON，XML相比可读性较差，但占用的空间也会更小，在一些对于速度要求比较高的场景中较为常用。

Java序列化Protocol Buffer框架—ProtoStuff

Google对于Protocol Buffer提供了多种语言的实现方法：Java，C++，go和python。但我们在使用时仍然需要去编写可读性不高的.proto文件，然后使用Google提供的实现方法编译成对应的语言，这就提高了我们使用Protocol Buffer的门槛。因此ProtoStuff就诞生了，通过ProtoStuff这个框架，我们能直接将对象通过Protocol Buffer这种序列化的方式转成对应的字节，极大地降低了我们使用Protocol Buffer的使用成本。

实例

首先我们新建一个maven项目，然后添加ProtoStuff的依赖，其中Objenesis是一个用来实例化一个特定类的新对象的Java库。通过该库，我们能在不调用构造函数的情况下实例化一个类的对象。

<dependency>
	<groupId>com.dyuproject.protostuff</groupId>
    <artifactId>protostuff-core</artifactId>
    <version>${protostuff.version}</version>
</dependency>

<dependency>
	<groupId>com.dyuproject.protostuff</groupId>
    <artifactId>protostuff-runtime</artifactId>
    <version>${protostuff.version}</version>
</dependency>

<!-- Objenesis -->
<dependency>
	<groupId>org.objenesis</groupId>
    <artifactId>objenesis</artifactId>
    <version>${objenesis.version}</version>
</dependency>
<!-- Lombok -->
<dependency>
	<groupId>org.projectlombok</groupId>
    <artifactId>lombok</artifactId>
    <version>${lombok.version}</version>
</dependency>

然后我们创建两个POJO来进行序列化的测试

@Data
@Builder
public Class Goods {
    
    private Integer num;
    private String name;
    private Double price;
    
}

@Data
@Builder
public Class Repository {
    
    private String name;
    private String location;
    private List<Goods> goodsList;
    
}

再之后编写Protocol Buffer序列化的工具类

public Class SerializationUtil {
    
    private static Map<Class<?>, Schema<?>> cacheSchema = new ConcurrentHashMap();
    private static Objenesis objenesis = new ObjenesisStd(true);
    
    /**
    * 序列化（对象 -> 字节数组）
    *
    */
    public static <T> byte[] serialize(T obj) {
        Class<T> cls = (Class<T>) obj.getClass();
        LinkedBuffer buffer = LinkedBuffer.allocate(LinkedBuffer.DEFAULT_BUFFER_SIZE);
        try {
            Schema<T> schema = getSchema(cls);
            return ProtobufIOUtil.toByteArray(obj, schema, buffer);
        } catch (Exception e) {
            throw new IllegalStateException(e.getMessage(), e);
        } finally {
            buffer.clear();
        }
    }
    
    /**
    * 反序列化（字节数组 -> 对象）
    *
    */
    public static <T> T deserilize(byte[] data, Class<T> cls) {
        try {
            T message = objenesis.newInstance(cls);
            Schema<T> schema = getSchema(cls);
            ProtobufIOUtil.mergeFrom(data, message, schema);
            return message;
        } catch (Exception e) {
            throw new IllegalStateException(e.getMessage(), e);
        }
    }
    
    @SuppressWarnings("unchecked")
    private static <T> Schema<T> getSchema(Class<T> cls) {
        Schema<T> schema = (Schema<T>) cacheSchema.get(cls);
        if (schema == null) {
            schema = RuntimeSchema.createFrom(cls);
            cacheSchema.put(cls, schema);
        }
        return schema;
    }
}

最后编写测试类来对序列化工具类进行测试

public Class Test {
    public static void main(String[] args) {
        Goods phone = Goods.builder().num(10).name("phone").price(1999.99).build();
        Goods water = Goods.builder().num(100).name("water").price(1.00).build();
        Repository repository = Repository.builder().name("Taobao").location("china").goodsList(Arrays.asList(phone, water)).build();
        byte[] data = SerializationUtil.serialize(repository);
        System.out.println("序列化结果：" + Arrays.toString(data));
        Repository result = SerializationUtil.deserilize(data, Repository.class);
        System.out.println("反序列化结果：" + result);
    }
}

输出结果：

序列化结果：[10, 6, 84, 97, 111, 98, 97, 111, 18, 5, 99, 104, 105, 110, 97, 26, 18, 8, 10, 18, 5, 112, 104, 111, 110, 101, 25, 41, 92, -113, -62, -11, 63, -97, 64, 26, 18, 8, 100, 18, 5, 119, 97, 116, 101, 114, 25, 0, 0, 0, 0, 0, 0, -16, 63]
反序列化结果：Repository(name=Taobao, location=china, goodsList=[Goods(num=10, name=phone, price=1999.99), Goods(num=100, name=water, price=1.0)])

与JSON的对比

首先导入JSON处理的依赖，这里我们使用jackson来对JSON进行处理

<dependency>
	<groupId>com.fasterxml.jackson.core</groupId>
    <artifactId>jackson-databind</artifactId>
    <version>${jackson.version}</version>
</dependency>

之后修改测试类

public class Test {
    public static void main(String[] args) throws IOException {
        Goods phone = Goods.builder().num(10).name("phone").price(1999.99).build();
        Goods water = Goods.builder().num(100).name("water").price(1.00).build();
        Repository repository = Repository.builder().name("Taobao").location("china").goodsList(Arrays.asList(phone, water)).build();

        byte[] protobufData = SerializationUtil.serialize(repository);
        System.out.println("ProtoBuf序列化结果：" + Arrays.toString(protobufData));
        Repository protobufResult = SerializationUtil.deserilize(protobufData, Repository.class);
        System.out.println("ProtoBuf反序列化结果：" + protobufResult);

        ObjectMapper mapper = new ObjectMapper();
        byte[] jsonData = mapper.writeValueAsBytes(repository);
        System.out.println("JSON序列化结果：" + Arrays.toString(jsonData));
        Repository jsonResult = mapper.readValue(jsonData, Repository.class);
        System.out.println("JSON序列化结果：" + jsonResult);

        System.out.println();
        System.out.println("ProtoBuf序列化后字符串结果：" + new String(protobufData, StandardCharsets.UTF_8));
        System.out.println("JSON序列化后字符串结果：" + new String(jsonData, StandardCharsets.UTF_8));

        System.out.println();
        System.out.println("ProtoBuf序列化长度：" + protobufData.length);
        System.out.println("JSON序列化长度：" + jsonData.length);
    }
}

输出结果：

ProtoBuf序列化结果：[10, 6, 84, 97, 111, 98, 97, 111, 18, 5, 99, 104, 105, 110, 97, 26, 18, 8, 10, 18, 5, 112, 104, 111, 110, 101, 25, 41, 92, -113, -62, -11, 63, -97, 64, 26, 18, 8, 100, 18, 5, 119, 97, 116, 101, 114, 25, 0, 0, 0, 0, 0, 0, -16, 63]
ProtoBuf反序列化结果：Repository(name=Taobao, location=china, goodsList=[Goods(num=10, name=phone, price=1999.99), Goods(num=100, name=water, price=1.0)])
JSON序列化结果：[123, 34, 110, 97, 109, 101, 34, 58, 34, 84, 97, 111, 98, 97, 111, 34, 44, 34, 108, 111, 99, 97, 116, 105, 111, 110, 34, 58, 34, 99, 104, 105, 110, 97, 34, 44, 34, 103, 111, 111, 100, 115, 76, 105, 115, 116, 34, 58, 91, 123, 34, 110, 117, 109, 34, 58, 49, 48, 44, 34, 110, 97, 109, 101, 34, 58, 34, 112, 104, 111, 110, 101, 34, 44, 34, 112, 114, 105, 99, 101, 34, 58, 49, 57, 57, 57, 46, 57, 57, 125, 44, 123, 34, 110, 117, 109, 34, 58, 49, 48, 48, 44, 34, 110, 97, 109, 101, 34, 58, 34, 119, 97, 116, 101, 114, 34, 44, 34, 112, 114, 105, 99, 101, 34, 58, 49, 46, 48, 125, 93, 125]
JSON序列化结果：Repository(name=Taobao, location=china, goodsList=[Goods(num=10, name=phone, price=1999.99), Goods(num=100, name=water, price=1.0)])

ProtoBuf序列化后字符串结果：
Taobaochina
phone)���?�@dwater �?
JSON序列化后字符串结果：{“name”:”Taobao”,”location”:”china”,”goodsList”:[{“num”:10,”name”:”phone”,”price”:1999.99},{“num”:100,”name”:”water”,”price”:1.0}]}

ProtoBuf序列化长度：55
JSON序列化长度：131

从结果来看在可读性上显然JSON更加易读，ProtoBuf序列化后再转为字符串甚至会乱码，但在长度上则显然ProtoBuf更占优势，JSON的长度比ProtoBuf多了一倍多。

⚠️：在使用Jackson进行JSON反序列化时我们需要对我们的POJO类添加有参和无参构造，即添加@NoArgsConstructor 和@AllArgsConstructor 这两个注解，否则会抛出如下异常：

Exception in thread “main” com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Cannot construct instance of com.xxx.xxx.Repository (no Creators, like default constructor, exist): cannot deserialize from Object value (no delegate- or property-based Creator)
at [Source: (byte[])”{“name”:”Taobao”,”location”:”china”,”goodsList”:[{“num”:10,”name”:”phone”,”price”:1999.99},{“num”:100,”name”:”water”,”price”:1.0}]}”; line: 1, column: 2]
at com.fasterxml.jackson.databind.exc.InvalidDefinitionException.from(InvalidDefinitionException.java:67)
at com.fasterxml.jackson.databind.DeserializationContext.reportBadDefinition(DeserializationContext.java:1764)
at com.fasterxml.jackson.databind.DatabindContext.reportBadDefinition(DatabindContext.java:400)
at com.fasterxml.jackson.databind.DeserializationContext.handleMissingInstantiator(DeserializationContext.java:1209)
at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1400)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:362)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:195)
at com.fasterxml.jackson.databind.deser.DefaultDeserializationContext.readRootValue(DefaultDeserializationContext.java:322)
at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4593)
at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3609)
at com.silence.rpc.test.Test.main(Test.java:31)

原因是因为@Builder并不会添加无参构造，而Jackson的反序列化需要无参构造，因为在反序列化的时候，会先初始化对象，此时默认调用的是无参函数，然后再进行赋值，故此我们需要添加@NoArgsConstructor ，如果只添加这个注解，又会导致缺少有参构造，因此我们还需要添加@AllArgsConstructor 。