由于要实现go服务端与嵌入式设备的通信打通,综合利弊选择golang与c++的grpc通信方式实现,GRPC由于原生不支持c语言(不可直接生成c的服务,但是可以生成序列化反序列化的代码,使用protoc-c),所以选用原生支持的c++,生成c++端的grpc序列化相关代码,grpc服务相关代码,至于grpc相关代码,若感兴趣可以试着自行尝试,但并不建议用在项目中,因为每次增加服务或者改变代码时,这部分都得自行做适配,且易出bug。
示例来源于官方代码的golang部分以及c++部分,实现现在的golang to c++、c++ to golang,至于前置的golang to golang可以看上两篇博文。
syntax = "proto3"; /* * 这里有一个坑,如果 option go_package="./;golang"; 没有设置,会出现下面的报错 * Please specify either: • a "go_package" option in the .proto source file, or • a "M" argument on the command line. */ option go_package="./;protofile"; option java_package = "ex.grpc"; package protofile; message Req { string message = 1; } message Res { string message = 1; } service HelloGRPC { rpc SayHi(Req) returns (Res); }
这个proto文件非常简单,定义了两个message,分别为Req(请求message)和Res(响应message),定义了一个service,service虽然只定义了一个服务函数SayHi,但是可以有多个不同的服务函数,而且service也可以同时存在多个,自然而然的message也是支持多个。
当message中存在多个元素时,通过等号后边的编号作区分,由于grpc得到的是二进制流,而且不像json那样是key:value对,解析时水到渠成,所以要通过给编号,在解析时通过编号做出区分,如果说json类似于map(golang)\dict(python),那么这种按照编号定位的则类似于slice(golang)\list(python),但是编号是从1开始,1到15为高频,后面为低频。
具体详细含义看上两篇博文,这里不赘述。
通过sh脚本(windows可以采用bst脚本)将命令写进去,可以快速进行编译。
protoc --go_out=. --go_opt=paths=source_relative --go-grpc_out=. --go-grpc_opt=paths=source_relative ./test.proto
protoc是官方的工具,如果想生成c代码可以使用protoc-c,但是无法提供grpc服务相关代码,只能生成序列化与反序列化代码。
主要的两个参数为 plugins 和 paths ,代表 生成 go 代码所使用的插件 和 生成的 go 代码的目录怎样架构。
paths 参数有两个选项,import和 source_relative。默认为 import ,代表按照生成的 go 代码的包的全路径去创建目录层级,source_relative 代表按照 proto 源文件的目录层级去创建 go 代码的目录层级,如果目录已存在则不用创建。
–go_out指定根据proto文件生成的后缀为.pb.go
文件(例如test.pb.go)存放的位置,命令中给.
,代表生成的文件就存放在本文件目录内。
–go_opt可以看做是对命令--go_out
的一个参数设置,这里设置paths=source_relative
。
这里的后缀为.pb.go
文件就是序列化与反序列化相关的代码,如果使用protoc-c生成c代码,就只能生成这个部分的c/h代码,而不能生成下面的部分
–go-grpc_out指定根据proto文件生成的后缀为.pb.go
的grpc服务相关文件例test_grpc.pb.go
存放的位置,命令中给.
,代表生成的文件就存放在本文件目录内。
–go-grpc_opt可以看做是对命令--go-grpc_out
的一个参数设置,这里设置paths=source_relative
。
golang服务端文件定义:
package main import ( "GRPC/protofile" "context" "fmt" "google.golang.org/grpc" "net" ) type server struct { protofile.UnimplementedHelloGRPCServer } func (s *server) SayHi(ctx context.Context, req *protofile.Req) (res *protofile.Res, err error){ fmt.Println(req.GetMessage()) return &protofile.Res{Message: "服务端响应"}, nil } func main(){ listen, _ := net.Listen("tcp", ":8000") s := grpc.NewServer() protofile.RegisterHelloGRPCServer(s, &server{}) s.Serve(listen) }
这里重点关注的几个点为:
1、 proto文件中定义的service会在生成的文件test_grpc.pb.go中做出struct定义:
Unimplemented+service名字(这里是HelloGRPC
)+Server
示例:
// UnimplementedHelloGRPCServer must be embedded to have forward compatible implementations. type UnimplementedHelloGRPCServer struct { }
然后在我们的服务端代码内,要定义一个struct,匿名继承这个struct,例如
type server struct { protofile.UnimplementedHelloGRPCServer }
2、proto文件中定义的service内部服务函数会在生成的文件test_grpc.pb.go中做出函数定义:
func (UnimplementedHelloGRPCServer) SayHi(context.Context, *Req) (*Res, error) { return nil, status.Errorf(codes.Unimplemented, "method SayHi not implemented") }
其实就是为上面提到的struct挂载方法,但是可以发现这个函数只是在初始阶段,即在服务里直接调用这个函数,也会正确返回,但是是初始值,而我们要在这里实现具体的服务,所以也要进行重写。
在服务端代码里做出重写:
func (s *server) SayHi(ctx context.Context, req *protofile.Req) (res *protofile.Res, err error){ fmt.Println(req.GetMessage()) return &protofile.Res{Message: "服务端响应"}, nil }
这里只做了简单的示例,真正开发时可以根据实际需要实现更复杂功能
3、 启动服务
利用net包,监听目标端口,协议设定为tcp。
listen, _ := net.Listen("tcp", ":8000")
新建grpc服务
s := grpc.NewServer()
初始化grpc服务
protofile.RegisterHelloGRPCServer(s, &server{})
启动服务
s.Serve(listen)
到这里就完成了golang端的grpc服务,可以发现,调用的几乎都是生成的test_grpc.pb.go内部函数,而test.pb.go中的序列化反序列化函数,都是在test_grpc.pb.go内部完成调用的,所以如果使用protoc-c生成代码时,test_grpc.pb.go内部的所有代码,都需要手动完成,目前代码量大概在100行左右,后期功能增加,功能变复杂时,其工作量可想而知,所以建议使用c++,因为这部分已经由protoc自动生成。
golang客户端文件定义:
package main import ( "GRPC/protofile" "context" "fmt" "google.golang.org/grpc" ) func main(){ conn,_ := grpc.Dial("localhost:8000", grpc.WithInsecure()) defer conn.Close() // 不这样做会一直无法关闭 client := protofile.NewHelloGRPCClient(conn) req, _ := client.SayHi(context.Background(), &protofile.Req{Message: "客户端消息"}) fmt.Println(req.GetMessage()) } 可以发现,所有代码目前都放在了main函数内,因为真的是太简单了,但是不排除后期随着功能复杂时,结构会有改变,但是原理是相同的。 这里同样有需要重点关注的几个点: 1、grpc拨号 ```go conn,_ := grpc.Dial("localhost:8000", grpc.WithInsecure())
这里因为做示例,所以暂时使用不安全模式grpc.WithInsecure()
2、链接关闭
defer conn.Close()// 不这样做会一直无法关闭
3、初始化client
client := protofile.NewHelloGRPCClient(conn)
4、调用远端服务函数
req, _ := client.SayHi(context.Background(), &protofile.Req{Message: "客户端消息"})
这里虽然不论感觉上还是效果上都和直接调用远端函数是一样的,但是实质上也是告诉远端要调用哪个函数,给这个函数哪些参数,然后数据压缩,最后远端接受到数据解析执行后,再告诉远端执行的是哪个函数,返回了怎样的响应,然后数据压缩上报,服务端接收后解析得到数据结果。
只不过这个过程都在grpc内部完成了。
到此完成了golang端的服务端与客户端其执行结果可以看上两篇博文。
c++端利用cmake得到makefile,再由makefile编译成可执行文件。
项目目录:
. ├── CMakeLists.txt ├── common.cmake ├── greeter_client.cc ├── greeter_server.cc └── protos └── protofile.proto
这里的protofile.proto内容和golang端的proto文件是相同的,在两端的proto必须确保完全相同!!!
,不然可能会造成调用时解析数据错乱出现异常不到的错误。
c++端主要参照了官方cmake进行的,common.cmake由官方项目得来,具体含义未研究,但不可缺少,CMakeLists.txt由官方项目中得来并根据实际情况进行改变。
greeter_client.cc与reeter_server.cc也根据官方项目和实际proto文件改变得到,重点关注内部创建grpc服务怎么实现。
定义:
# Copyright 2018 gRPC authors. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. # # cmake build file for C++ route_guide example. # Assumes protobuf and gRPC have been installed using cmake. # See cmake_externalproject/CMakeLists.txt for all-in-one cmake build # that automatically builds all the dependencies before building route_guide. cmake_minimum_required(VERSION 3.5.1) set (CMAKE_CXX_STANDARD 11) if(MSVC) add_definitions(-D_WIN32_WINNT=0x600) endif() find_package(Threads REQUIRED) if(GRPC_AS_SUBMODULE) # One way to build a projects that uses gRPC is to just include the # entire gRPC project tree via "add_subdirectory". # This approach is very simple to use, but the are some potential # disadvantages: # * it includes gRPC's CMakeLists.txt directly into your build script # without and that can make gRPC's internal setting interfere with your # own build. # * depending on what's installed on your system, the contents of submodules # in gRPC's third_party/* might need to be available (and there might be # additional prerequisites required to build them). Consider using # the gRPC_*_PROVIDER options to fine-tune the expected behavior. # # A more robust approach to add dependency on gRPC is using # cmake's ExternalProject_Add (see cmake_externalproject/CMakeLists.txt). # Include the gRPC's cmake build (normally grpc source code would live # in a git submodule called "third_party/grpc", but this example lives in # the same repository as gRPC sources, so we just look a few directories up) add_subdirectory(../../.. ${CMAKE_CURRENT_BINARY_DIR}/grpc EXCLUDE_FROM_ALL) message(STATUS "Using gRPC via add_subdirectory.") # After using add_subdirectory, we can now use the grpc targets directly from # this build. set(_PROTOBUF_LIBPROTOBUF libprotobuf) set(_REFLECTION grpc++_reflection) if(CMAKE_CROSSCOMPILING) find_program(_PROTOBUF_PROTOC protoc) else() set(_PROTOBUF_PROTOC $<TARGET_FILE:protobuf::protoc>) endif() set(_GRPC_GRPCPP grpc++) if(CMAKE_CROSSCOMPILING) find_program(_GRPC_CPP_PLUGIN_EXECUTABLE grpc_cpp_plugin) else() set(_GRPC_CPP_PLUGIN_EXECUTABLE $<TARGET_FILE:grpc_cpp_plugin>) endif() elseif(GRPC_FETCHCONTENT) # Another way is to use CMake's FetchContent module to clone gRPC at # configure time. This makes gRPC's source code available to your project, # similar to a git submodule. message(STATUS "Using gRPC via add_subdirectory (FetchContent).") include(FetchContent) FetchContent_Declare( grpc GIT_REPOSITORY https://github.com/grpc/grpc.git # when using gRPC, you will actually set this to an existing tag, such as # v1.25.0, v1.26.0 etc.. # For the purpose of testing, we override the tag used to the commit # that's currently under test. GIT_TAG vGRPC_TAG_VERSION_OF_YOUR_CHOICE) FetchContent_MakeAvailable(grpc) # Since FetchContent uses add_subdirectory under the hood, we can use # the grpc targets directly from this build. set(_PROTOBUF_LIBPROTOBUF libprotobuf) set(_REFLECTION grpc++_reflection) set(_PROTOBUF_PROTOC $<TARGET_FILE:protoc>) set(_GRPC_GRPCPP grpc++) if(CMAKE_CROSSCOMPILING) find_program(_GRPC_CPP_PLUGIN_EXECUTABLE grpc_cpp_plugin) else() set(_GRPC_CPP_PLUGIN_EXECUTABLE $<TARGET_FILE:grpc_cpp_plugin>) endif() else() # This branch assumes that gRPC and all its dependencies are already installed # on this system, so they can be located by find_package(). # Find Protobuf installation # Looks for protobuf-config.cmake file installed by Protobuf's cmake installation. set(protobuf_MODULE_COMPATIBLE TRUE) find_package(Protobuf CONFIG REQUIRED) message(STATUS "Using protobuf ${Protobuf_VERSION}") set(_PROTOBUF_LIBPROTOBUF protobuf::libprotobuf) set(_REFLECTION gRPC::grpc++_reflection) if(CMAKE_CROSSCOMPILING) find_program(_PROTOBUF_PROTOC protoc) else() set(_PROTOBUF_PROTOC $<TARGET_FILE:protobuf::protoc>) endif() # Find gRPC installation # Looks for gRPCConfig.cmake file installed by gRPC's cmake installation. find_package(gRPC CONFIG REQUIRED) message(STATUS "Using gRPC ${gRPC_VERSION}") set(_GRPC_GRPCPP gRPC::grpc++) if(CMAKE_CROSSCOMPILING) find_program(_GRPC_CPP_PLUGIN_EXECUTABLE grpc_cpp_plugin) else() set(_GRPC_CPP_PLUGIN_EXECUTABLE $<TARGET_FILE:gRPC::grpc_cpp_plugin>) endif() endif()
这个文件直接复制就可以,不需要改变,感兴趣可以自行研究。
# Copyright 2018 gRPC authors. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. # # cmake build file for C++ helloworld example. # Assumes protobuf and gRPC have been installed using cmake. # See cmake_externalproject/CMakeLists.txt for all-in-one cmake build # that automatically builds all the dependencies before building helloworld. cmake_minimum_required(VERSION 3.5.1) project(Protofile C CXX) include(./common.cmake) # Proto file get_filename_component(ty_proto "protos/protofile.proto" ABSOLUTE) get_filename_component(ty_proto_path "${ty_proto}" PATH) # Generated sources set(ty_proto_srcs "${CMAKE_CURRENT_BINARY_DIR}/protofile.pb.cc") set(ty_proto_hdrs "${CMAKE_CURRENT_BINARY_DIR}/protofile.pb.h") set(ty_grpc_srcs "${CMAKE_CURRENT_BINARY_DIR}/protofile.grpc.pb.cc") set(ty_grpc_hdrs "${CMAKE_CURRENT_BINARY_DIR}/protofile.grpc.pb.h") add_custom_command( OUTPUT "${ty_proto_srcs}" "${ty_proto_hdrs}" "${ty_grpc_srcs}" "${ty_grpc_hdrs}" COMMAND ${_PROTOBUF_PROTOC} ARGS --grpc_out "${CMAKE_CURRENT_BINARY_DIR}" --cpp_out "${CMAKE_CURRENT_BINARY_DIR}" -I "${ty_proto_path}" --plugin=protoc-gen-grpc="${_GRPC_CPP_PLUGIN_EXECUTABLE}" "${ty_proto}" DEPENDS "${ty_proto}") # Include generated *.pb.h files include_directories("${CMAKE_CURRENT_BINARY_DIR}") # ty_grpc_proto add_library(ty_grpc_proto ${ty_grpc_srcs} ${ty_grpc_hdrs} ${ty_proto_srcs} ${ty_proto_hdrs}) target_link_libraries(ty_grpc_proto ${_REFLECTION} ${_GRPC_GRPCPP} ${_PROTOBUF_LIBPROTOBUF}) # Targets greeter_[async_](client|server) foreach(_target greeter_client greeter_server ) add_executable(${_target} "${_target}.cc") target_link_libraries(${_target} ty_grpc_proto ${_REFLECTION} ${_GRPC_GRPCPP} ${_PROTOBUF_LIBPROTOBUF}) endforeach()
本人对cmake不是太了解,就不好详细叙述了,但是仔细对照下目录文件和代码,我觉得几乎所有人都是可以抄出来自己的cmake文件的。
使用cmake以下命令构建示例:
$ mkdir -p cmake/build $ pushd cmake/build $ cmake -DCMAKE_PREFIX_PATH=$MY_INSTALL_DIR ../.. $ make -j
$MY_INSTALL_DIR替换成自己grpc安装目录,例如我的安状在/usr/local/gRPC,则这里的命令为:
$ cmake -DCMAKE_PREFIX_PATH=$MY_INSTALL_DIR ../..
文件定义:
/* * * Copyright 2015 gRPC authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * * http://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. * See the License for the specific language governing permissions and * limitations under the License. * */ #include <iostream> #include <memory> #include <string> #include <grpcpp/ext/proto_server_reflection_plugin.h> #include <grpcpp/grpcpp.h> #include <grpcpp/health_check_service_interface.h> #ifdef BAZEL_BUILD #include "examples/protos/protofile.grpc.pb.h" #else #include "protofile.grpc.pb.h" #endif using grpc::Server; using grpc::ServerBuilder; using grpc::ServerContext; using grpc::Status; using protofile::Req; // 请求 using protofile::Res; // 响应 using protofile::HelloGRPC; // 服务 // Logic and data behind the server's behavior. class HelloGRPCServiceImpl final : public HelloGRPC::Service { Status SayHi(ServerContext* context, const Req* request, Res* reply) { std::string prefix("Hello "); reply->set_message(prefix + request->message()); return Status::OK; } }; void RunServer() { std::string server_address("0.0.0.0:8000"); HelloGRPCServiceImpl service; grpc::EnableDefaultHealthCheckService(true); grpc::reflection::InitProtoReflectionServerBuilderPlugin(); ServerBuilder builder; // Listen on the given address without any authentication mechanism. builder.AddListeningPort(server_address, grpc::InsecureServerCredentials()); // Register "service" as the instance through which we'll communicate with // clients. In this case it corresponds to an *synchronous* service. builder.RegisterService(&service); // Finally assemble the server. std::unique_ptr<Server> server(builder.BuildAndStart()); std::cout << "Server listening on " << server_address << std::endl; // Wait for the server to shutdown. Note that some other thread must be // responsible for shutting down the server for this call to ever return. server->Wait(); } int main(int argc, char** argv) { RunServer(); return 0; }
文件定义:
/* * * Copyright 2015 gRPC authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * * http://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. * See the License for the specific language governing permissions and * limitations under the License. * */ #include <iostream> #include <memory> #include <string> #include <grpcpp/grpcpp.h> #ifdef BAZEL_BUILD #include "examples/protos/protofile.grpc.pb.h" #else #include "protofile.grpc.pb.h" #endif using grpc::Channel; using grpc::ClientContext; using grpc::Status; using protofile::Req; // 请求 using protofile::Res; // 响应 using protofile::HelloGRPC; // 服务 class HelloGRPCClient { public: HelloGRPCClient(std::shared_ptr<Channel> channel) : stub_(HelloGRPC::NewStub(channel)) {} // Assembles the client's payload, sends it and presents the response back // from the server. std::string SayHi(const std::string& user) { // Data we are sending to the server. Req request; request.set_message(user); // Container for the data we expect from the server. Res reply; // Context for the client. It could be used to convey extra information to // the server and/or tweak certain RPC behaviors. ClientContext context; // The actual RPC. Status status = stub_->SayHi(&context, request, &reply); // Act upon its status. if (status.ok()) { return reply.message(); } else { std::cout << status.error_code() << ": " << status.error_message() << std::endl; return "RPC failed"; } } private: std::unique_ptr<HelloGRPC::Stub> stub_; }; int main(int argc, char** argv) { // Instantiate the client. It requires a channel, out of which the actual RPCs // are created. This channel models a connection to an endpoint specified by // the argument "--target=" which is the only expected argument. // We indicate that the channel isn't authenticated (use of // InsecureChannelCredentials()). std::string target_str; std::string arg_str("--target"); if (argc > 1) { std::string arg_val = argv[1]; size_t start_pos = arg_val.find(arg_str); if (start_pos != std::string::npos) { start_pos += arg_str.size(); if (arg_val[start_pos] == '=') { target_str = arg_val.substr(start_pos + 1); } else { std::cout << "The only correct argument syntax is --target=" << std::endl; return 0; } } else { std::cout << "The only acceptable argument is --target=" << std::endl; return 0; } } else { target_str = "localhost:8000"; } HelloGRPCClient greeter( grpc::CreateChannel(target_str, grpc::InsecureChannelCredentials())); std::string user("world"); std::string reply = greeter.SayHi(user); std::cout << "Greeter received: " << reply << std::endl; return 0; }
c++不是太了解,但是流程与golang是相同的,都是_grpc.pb
文件内根据proto文件生成的grpc服务函数的各种struct(golang中为struct,进行struct继承,挂载方法)与class(c++中为class,进行class继承,class函数重写),的重写过程,然后调用从proto文件生成的_grpc.pb
文件中的服务。
个人比较习惯c,c++有些抵触,而且由于使用受体是c语言开发,所以计划是采用c++对外开放指针函数,外界在实现具体服务函数,然后注册到c++对应开放的指针函数。
具体效果就不演示了,但是这个已经是调通的示例了,拿过去可以直接运行。
至于golang与c++的配置gRPC环境过程,参考上两篇gRPC文章。