Fwd: Protobuf string serialization bug with statically linked protobuf 2.5.0

Fwd: Protobuf string serialization bug with statically linked protobuf 2.5.0

[Tomasz Wiszkowski]

Dear all,

I'm having problems with statically linked executables that use
protocol buffers.
I suspect the problem may be related to incompatibility between
std::string implementation used to compile the library vs. current. If
that's the case, the problem would likely go away with recompilation
of the protocol buffer libraries (protobuf-lite is also exposing the
same problem).

I have attached a test case as you requested. the example program
compiles two variants - one dynamically linked (works fine) and one
statically linked that crashes upon first attempt to serialize the
protocol buffer.

It would be great if someone could take a look and possibly rebuild
the static libraries for protocol buffers.

Best regards,
Tomasz

-------- example.proto ------------
syntax = "proto2";

package example;

message ExampleMsg {
  optional int32 argc = 1;
  optional string argv0 = 2;
};

-------- main.cc ------------
#include <iostream>
#include <string>

#include "example.pb.h"

int main(int argc, char** argv) {
  example::ExampleMsg message;

  message.set_argc(argc);
  message.set_argv0(argv[0]);

  std::cout << "Serializing protocol buffer." << std::endl;
  std::string serialized;
  message.SerializeToString(&serialized);  // static variant crashes here.
  std::cout << "Serialized length: " << serialized.length() << std::endl;

  message.Clear();

  std::cout << "Deserializing protocol buffer." << std::endl;
  message.ParseFromString(serialized);  // static variant also crashes here.
  std::cout << "Deserialized content: argc=" << message.argc() << ", argv0="
            << message.argv0();

  return 0;
}

-------- Makefile ------------
CFLAGS += -Wall
CXXFLAGS := $(CFLAGS)
CC = g++
LIBS = -lprotobuf.dll

all: clean example example-bug

example.pb.cc: example.proto
        protoc --cpp_out=. $^

clean:
        rm -f *.o *.pb.* *.exe*

example: example.pb.o main.o
        $(CC) $(CFLAGS) $^ -o $@ $(LIBS)

example-bug: example.pb.o main.o
        $(CC) $(CFLAGS) -static $^ -o $@ $(LIBS)

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Protobuf string serialization bug with statically linked protobuf 2.5.0

[Tomasz Wiszkowski]

A brief correction.

I tried to identify further the problem by selectively linking
libraries. (-Wl,-Bstatic -lprotobuf.dll -Wl,-Bdynamic) and it turns
out the problem emerges the moment i statically link both protobuf
_and_ the c++ library. linking everything statically except for libc++
seems to work just fine.

i checked if protobuf loads cygc++-6 separately, which would easily
explain the problem, but it does not seem to. the problem clearly
seems to be interaction between these two.

linking libc++ dynamically resolves the problem, but that would also
mean shipping the software with artificial add-on... i suspect
recompilation still might resolve the issue...


--
"My definition of an expert in any field is a person who knows enough
about what's really going on to be scared" - P. J. Plauger


2016-02-10 10:54 GMT-08:00 Tomasz Wiszkowski <tomasz.wiszkowski@gmail.com>:
> Dear all,
>
> I'm having problems with statically linked executables that use
> protocol buffers.
> I suspect the problem may be related to incompatibility between
> std::string implementation used to compile the library vs. current. If
> that's the case, the problem would likely go away with recompilation
> of the protocol buffer libraries (protobuf-lite is also exposing the
> same problem).
>
> I have attached a test case as you requested. the example program
> compiles two variants - one dynamically linked (works fine) and one
> statically linked that crashes upon first attempt to serialize the
> protocol buffer.
>
> It would be great if someone could take a look and possibly rebuild
> the static libraries for protocol buffers.
>
> Best regards,
> Tomasz
>
> -------- example.proto ------------
> syntax = "proto2";
>
> package example;
>
> message ExampleMsg {
>   optional int32 argc = 1;
>   optional string argv0 = 2;
> };
>
> -------- main.cc ------------
> #include <iostream>
> #include <string>
>
> #include "example.pb.h"
>
> int main(int argc, char** argv) {
>   example::ExampleMsg message;
>
>   message.set_argc(argc);
>   message.set_argv0(argv[0]);
>
>   std::cout << "Serializing protocol buffer." << std::endl;
>   std::string serialized;
>   message.SerializeToString(&serialized);  // static variant crashes here.
>   std::cout << "Serialized length: " << serialized.length() << std::endl;
>
>   message.Clear();
>
>   std::cout << "Deserializing protocol buffer." << std::endl;
>   message.ParseFromString(serialized);  // static variant also crashes here.
>   std::cout << "Deserialized content: argc=" << message.argc() << ", argv0="
>             << message.argv0();
>
>   return 0;
> }
>
> -------- Makefile ------------
> CFLAGS += -Wall
> CXXFLAGS := $(CFLAGS)
> CC = g++
> LIBS = -lprotobuf.dll
>
> all: clean example example-bug
>
> example.pb.cc: example.proto
>         protoc --cpp_out=. $^
>
> clean:
>         rm -f *.o *.pb.* *.exe*
>
> example: example.pb.o main.o
>         $(CC) $(CFLAGS) $^ -o $@ $(LIBS)
>
> example-bug: example.pb.o main.o
>         $(CC) $(CFLAGS) -static $^ -o $@ $(LIBS)

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Protobuf string serialization bug with statically linked protobuf 2.5.0

[Tomasz Wiszkowski]

I gather this package bug is not exactly interesting here, but i'll
share a final update.

The package is likely built inaccurately, or is linked against library
that changed ABI over time.

Downloading and compiling protocol buffers library manually does not
expose any of the serialization problems.
furthermore, manual compilation does not introduce an artificial
dependency on cygprotobuf.dll library.

the protobuf package needs to be rebuilt.

thank you for your attention, i will not be disturbing the list with
further updates.

--
"My definition of an expert in any field is a person who knows enough
about what's really going on to be scared" - P. J. Plauger


2016-02-11 15:20 GMT-08:00 Tomasz Wiszkowski <tomasz.wiszkowski@gmail.com>:
> A brief correction.
>
> I tried to identify further the problem by selectively linking
> libraries. (-Wl,-Bstatic -lprotobuf.dll -Wl,-Bdynamic) and it turns
> out the problem emerges the moment i statically link both protobuf
> _and_ the c++ library. linking everything statically except for libc++
> seems to work just fine.
>
> i checked if protobuf loads cygc++-6 separately, which would easily
> explain the problem, but it does not seem to. the problem clearly
> seems to be interaction between these two.
>
> linking libc++ dynamically resolves the problem, but that would also
> mean shipping the software with artificial add-on... i suspect
> recompilation still might resolve the issue...
>
>
> --
> "My definition of an expert in any field is a person who knows enough
> about what's really going on to be scared" - P. J. Plauger
>
>
> 2016-02-10 10:54 GMT-08:00 Tomasz Wiszkowski <tomasz.wiszkowski@gmail.com>:
>> Dear all,
>>
>> I'm having problems with statically linked executables that use
>> protocol buffers.
>> I suspect the problem may be related to incompatibility between
>> std::string implementation used to compile the library vs. current. If
>> that's the case, the problem would likely go away with recompilation
>> of the protocol buffer libraries (protobuf-lite is also exposing the
>> same problem).
>>
>> I have attached a test case as you requested. the example program
>> compiles two variants - one dynamically linked (works fine) and one
>> statically linked that crashes upon first attempt to serialize the
>> protocol buffer.
>>
>> It would be great if someone could take a look and possibly rebuild
>> the static libraries for protocol buffers.
>>
>> Best regards,
>> Tomasz
>>
>> -------- example.proto ------------
>> syntax = "proto2";
>>
>> package example;
>>
>> message ExampleMsg {
>>   optional int32 argc = 1;
>>   optional string argv0 = 2;
>> };
>>
>> -------- main.cc ------------
>> #include <iostream>
>> #include <string>
>>
>> #include "example.pb.h"
>>
>> int main(int argc, char** argv) {
>>   example::ExampleMsg message;
>>
>>   message.set_argc(argc);
>>   message.set_argv0(argv[0]);
>>
>>   std::cout << "Serializing protocol buffer." << std::endl;
>>   std::string serialized;
>>   message.SerializeToString(&serialized);  // static variant crashes here.
>>   std::cout << "Serialized length: " << serialized.length() << std::endl;
>>
>>   message.Clear();
>>
>>   std::cout << "Deserializing protocol buffer." << std::endl;
>>   message.ParseFromString(serialized);  // static variant also crashes here.
>>   std::cout << "Deserialized content: argc=" << message.argc() << ", argv0="
>>             << message.argv0();
>>
>>   return 0;
>> }
>>
>> -------- Makefile ------------
>> CFLAGS += -Wall
>> CXXFLAGS := $(CFLAGS)
>> CC = g++
>> LIBS = -lprotobuf.dll
>>
>> all: clean example example-bug
>>
>> example.pb.cc: example.proto
>>         protoc --cpp_out=. $^
>>
>> clean:
>>         rm -f *.o *.pb.* *.exe*
>>
>> example: example.pb.o main.o
>>         $(CC) $(CFLAGS) $^ -o $@ $(LIBS)
>>
>> example-bug: example.pb.o main.o
>>         $(CC) $(CFLAGS) -static $^ -o $@ $(LIBS)

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: Protobuf string serialization bug with statically linked protobuf 2.5.0

[Achim Gratz]

[please don't top-post]

Tomasz Wiszkowski writes:
> the protobuf package needs to be rebuilt.

That package is orphaned.  Since you already have it built and are
obviously using it, would you like to maintain it?


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptation for Waldorf microQ V2.22R2:
http://Synth.Stromeko.net/Downloads.html#WaldorfSDada

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple