Commit ca65dfc7 authored by Bas Couwenberg's avatar Bas Couwenberg

New upstream version 1.6.1

parent 3d1e6529
......@@ -13,7 +13,7 @@ Checks: '*,-cert-dcl21-cpp,-cert-err60-cpp,-cppcoreguidelines-pro-bound
# This is a low-level library, it needs to do pointer arithmetic.
#
# cppcoreguidelines-pro-bounds-array-to-pointer-decay
# Limited use and many false positives including all for all asserts
# Limited use and many false positives including for all asserts
#
# cppcoreguidelines-pro-type-reinterpret-cast
# This is a low-level library, it needs to do reinterpret-casts.
......
......@@ -5,6 +5,39 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](http://keepachangelog.com/)
This project adheres to [Semantic Versioning](http://semver.org/).
## [unreleased] -
### Added
### Changed
### Fixed
## [1.6.1] - 2017-11-16
### Added
- Document internal handling of varints.
- Add aliases for fixed iterators, too.
### Changed
- The `const_fixed_iterator` is now a random access iterator making code
using it potentially more performant (for instance when using
`std::distance`)
- Overloads `std::distance` for the varint and svarint iterators. This is
better than the workaround with the `rage_size` function used before.
### Fixed
- Rename `.proto` files in some tests to be unique. This solves a problem
when building with newer versions of the Google Protobuf library.
- Floating point comparisons in tests are now always correctly done using
`Approx()`.
## [1.6.0] - 2017-10-24
### Added
......@@ -255,7 +288,8 @@ This project adheres to [Semantic Versioning](http://semver.org/).
- Make pbf reader and writer code endianess-aware.
[unreleased]: https://github.com/osmcode/libosmium/compare/v1.6.0...HEAD
[unreleased]: https://github.com/osmcode/libosmium/compare/v1.6.1...HEAD
[1.6.1]: https://github.com/osmcode/libosmium/compare/v1.6.0...v1.6.1
[1.6.0]: https://github.com/osmcode/libosmium/compare/v1.5.3...v1.6.0
[1.5.3]: https://github.com/osmcode/libosmium/compare/v1.5.2...v1.5.3
[1.5.2]: https://github.com/osmcode/libosmium/compare/v1.5.1...v1.5.2
......
......@@ -14,7 +14,7 @@ project(protozero)
set(PROTOZERO_VERSION_MAJOR 1)
set(PROTOZERO_VERSION_MINOR 6)
set(PROTOZERO_VERSION_PATCH 0)
set(PROTOZERO_VERSION_PATCH 1)
set(PROTOZERO_VERSION
"${PROTOZERO_VERSION_MAJOR}.${PROTOZERO_VERSION_MINOR}.${PROTOZERO_VERSION_PATCH}")
......
......@@ -14,6 +14,7 @@ the Google Protobufs `protoc` program.
[![Travis Build Status](https://travis-ci.org/mapbox/protozero.svg?branch=master)](https://travis-ci.org/mapbox/protozero)
[![Appveyor Build Status](https://ci.appveyor.com/api/projects/status/github/mapbox/protozero?svg=true)](https://ci.appveyor.com/project/Mapbox/protozero)
[![Coverage Status](https://codecov.io/gh/mapbox/protozero/branch/master/graph/badge.svg)](https://codecov.io/gh/mapbox/protozero)
[![Packaging status](https://repology.org/badge/tiny-repos/protozero.svg)](https://repology.org/metapackage/protozero)
## Depends
......
......@@ -8,11 +8,11 @@ bash -lc "pacman -S --needed --noconfirm mingw-w64-x86_64-gcc mingw-w64-x86_64-c
echo "Generating makefiles"
mkdir build
cd build
cmake .. -G "MSYS Makefiles"
cmake .. -LA -G "MSYS Makefiles"
echo "Building"
make
make VERBOSE=1
echo "Testing"
ctest
ctest --output-on-failure
......@@ -224,6 +224,26 @@ fields.
The function is also available in the `pbf_builder` class.
## Internal handling of varints
When varints are decoded they are always decoded as 64bit unsigned integers and
after that casted to the type you are requesting (using `static_cast`). This
means that if the protocol buffer message was created with a different integer
type than what you are reading it with, you might get wrong results without any
warning or error. This is the same behaviour as the Google Protocol Buffers
library has.
In normal use, this should never matter, because presumably you are using the
same types to write that data as you are using to read it later. It can happen
if the data is corrupted intentionally or unintentionally in some way. But
this can't be used to feed you any data that it wasn't possible to feed you
without this behaviour, so it doesn't open up any potential problems. You
always have to check anyway that the integers are in the range you expected
them to be in if the expected range is different than the range of the integer
type. This is especially true for enums which protozero will return as
`int32_t`.
## How many items are there in a repeated packed field?
Sometimes it is useful to know how many values there are in a repeated packed
......@@ -245,8 +265,7 @@ for (auto value : range) {
It depends on the type of range how expensive the `size()` call is. For ranges
derived from packed repeated fixed sized values the effort will be constant,
for ranges derived from packed repeated varints, the effort will be linear, but
still considerably cheaper than decoding the varints (for instance by calling
`std::distance(range.begin(), range.end());`). You have to benchmark your use
case to see whether the `reserve()` (or whatever you are using the `size()`
for) is worth it.
still considerably cheaper than decoding the varints. You have to benchmark
your use case to see whether the `reserve()` (or whatever you are using the
`size()` for) is worth it.
......@@ -62,8 +62,8 @@ public:
/**
* Create iterator range from two iterators.
*
* @param first_iterator Iterator to beginning or range.
* @param last_iterator Iterator to end or range.
* @param first_iterator Iterator to beginning of range.
* @param last_iterator Iterator to end of range.
*/
constexpr iterator_range(iterator&& first_iterator, iterator&& last_iterator) :
P(std::forward<iterator>(first_iterator),
......@@ -105,7 +105,7 @@ public:
* Complexity: Constant or linear depending on the underlaying iterator.
*/
std::size_t size() const noexcept {
return T::range_size(begin(), end());
return static_cast<size_t>(std::distance(begin(), end()));
}
/**
......@@ -164,16 +164,12 @@ class const_fixed_iterator {
public:
using iterator_category = std::forward_iterator_tag;
using iterator_category = std::random_access_iterator_tag;
using value_type = T;
using difference_type = std::ptrdiff_t;
using pointer = value_type*;
using reference = value_type&;
static std::size_t range_size(const const_fixed_iterator& begin, const const_fixed_iterator& end) noexcept {
return static_cast<std::size_t>(end.m_data - begin.m_data) / sizeof(T);
}
const_fixed_iterator() noexcept = default;
explicit const_fixed_iterator(const char* data) noexcept :
......@@ -197,25 +193,89 @@ public:
return result;
}
const_fixed_iterator& operator++() {
const_fixed_iterator& operator++() noexcept {
m_data += sizeof(value_type);
return *this;
}
const_fixed_iterator operator++(int) {
const_fixed_iterator operator++(int) noexcept {
const const_fixed_iterator tmp{*this};
++(*this);
return tmp;
}
bool operator==(const const_fixed_iterator& rhs) const noexcept {
bool operator==(const_fixed_iterator rhs) const noexcept {
return m_data == rhs.m_data;
}
bool operator!=(const const_fixed_iterator& rhs) const noexcept {
bool operator!=(const_fixed_iterator rhs) const noexcept {
return !(*this == rhs);
}
const_fixed_iterator& operator--() noexcept {
m_data -= sizeof(value_type);
return *this;
}
const_fixed_iterator operator--(int) noexcept {
const const_fixed_iterator tmp{*this};
--(*this);
return tmp;
}
friend bool operator<(const_fixed_iterator lhs, const_fixed_iterator rhs) noexcept {
return lhs.m_data < rhs.m_data;
}
friend bool operator>(const_fixed_iterator lhs, const_fixed_iterator rhs) noexcept {
return rhs < lhs;
}
friend bool operator<=(const_fixed_iterator lhs, const_fixed_iterator rhs) noexcept {
return !(lhs > rhs);
}
friend bool operator>=(const_fixed_iterator lhs, const_fixed_iterator rhs) noexcept {
return !(lhs < rhs);
}
const_fixed_iterator& operator+=(difference_type val) noexcept {
m_data += (sizeof(value_type) * val);
return *this;
}
friend const_fixed_iterator operator+(const_fixed_iterator lhs, difference_type rhs) noexcept {
const_fixed_iterator tmp{lhs};
tmp.m_data += (sizeof(value_type) * rhs);
return tmp;
}
friend const_fixed_iterator operator+(difference_type lhs, const_fixed_iterator rhs) noexcept {
const_fixed_iterator tmp{rhs};
tmp.m_data += (sizeof(value_type) * lhs);
return tmp;
}
const_fixed_iterator& operator-=(difference_type val) noexcept {
m_data -= (sizeof(value_type) * val);
return *this;
}
friend const_fixed_iterator operator-(const_fixed_iterator lhs, difference_type rhs) noexcept {
const_fixed_iterator tmp{lhs};
tmp.m_data -= (sizeof(value_type) * rhs);
return tmp;
}
friend difference_type operator-(const_fixed_iterator lhs, const_fixed_iterator rhs) noexcept {
return static_cast<difference_type>(lhs.m_data - rhs.m_data) / static_cast<difference_type>(sizeof(T));
}
value_type operator[](difference_type n) const noexcept {
return *(*this + n);
}
}; // class const_fixed_iterator
/**
......@@ -241,13 +301,13 @@ public:
using pointer = value_type*;
using reference = value_type&;
static std::size_t range_size(const const_varint_iterator& begin, const const_varint_iterator& end) noexcept {
static difference_type distance(const_varint_iterator begin, const_varint_iterator end) noexcept {
// We know that each varint contains exactly one byte with the most
// significant bit not set. We can use this to quickly figure out
// how many varints there are without actually decoding the varints.
return static_cast<std::size_t>(std::count_if(begin.m_data, end.m_data, [](char c) {
return std::count_if(begin.m_data, end.m_data, [](char c) noexcept {
return (static_cast<unsigned char>(c) & 0x80) == 0;
}));
});
}
const_varint_iterator() noexcept = default;
......@@ -342,4 +402,54 @@ public:
} // end namespace protozero
namespace std {
// Specialize std::distance for all the protozero iterators. Because
// functions can't be partially specialized, we have to do this for
// every value_type we are using.
template <>
inline typename protozero::const_varint_iterator<int32_t>::difference_type
distance<protozero::const_varint_iterator<int32_t>>(protozero::const_varint_iterator<int32_t> first, // NOLINT clang-tidy: readability-inconsistent-declaration-parameter-name
protozero::const_varint_iterator<int32_t> last) {
return protozero::const_varint_iterator<int32_t>::distance(first, last);
}
template <>
inline typename protozero::const_varint_iterator<int64_t>::difference_type
distance<protozero::const_varint_iterator<int64_t>>(protozero::const_varint_iterator<int64_t> first, // NOLINT clang-tidy: readability-inconsistent-declaration-parameter-name
protozero::const_varint_iterator<int64_t> last) {
return protozero::const_varint_iterator<int64_t>::distance(first, last);
}
template <>
inline typename protozero::const_varint_iterator<uint32_t>::difference_type
distance<protozero::const_varint_iterator<uint32_t>>(protozero::const_varint_iterator<uint32_t> first, // NOLINT clang-tidy: readability-inconsistent-declaration-parameter-name
protozero::const_varint_iterator<uint32_t> last) {
return protozero::const_varint_iterator<uint32_t>::distance(first, last);
}
template <>
inline typename protozero::const_varint_iterator<uint64_t>::difference_type
distance<protozero::const_varint_iterator<uint64_t>>(protozero::const_varint_iterator<uint64_t> first, // NOLINT clang-tidy: readability-inconsistent-declaration-parameter-name
protozero::const_varint_iterator<uint64_t> last) {
return protozero::const_varint_iterator<uint64_t>::distance(first, last);
}
template <>
inline typename protozero::const_svarint_iterator<int32_t>::difference_type
distance<protozero::const_svarint_iterator<int32_t>>(protozero::const_svarint_iterator<int32_t> first, // NOLINT clang-tidy: readability-inconsistent-declaration-parameter-name
protozero::const_svarint_iterator<int32_t> last) {
return protozero::const_svarint_iterator<int32_t>::distance(first, last);
}
template <>
inline typename protozero::const_svarint_iterator<int64_t>::difference_type
distance<protozero::const_svarint_iterator<int64_t>>(protozero::const_svarint_iterator<int64_t> first, // NOLINT clang-tidy: readability-inconsistent-declaration-parameter-name
protozero::const_svarint_iterator<int64_t> last) {
return protozero::const_svarint_iterator<int64_t>::distance(first, last);
}
} // end namespace std
#endif // PROTOZERO_ITERATORS_HPP
......@@ -745,6 +745,24 @@ public:
/// Forward iterator for iterating over uint64 (varint) values.
using const_uint64_iterator = const_varint_iterator<uint64_t>;
/// Forward iterator for iterating over fixed32 values.
using const_fixed32_iterator = const_fixed_iterator<uint32_t>;
/// Forward iterator for iterating over sfixed32 values.
using const_sfixed32_iterator = const_fixed_iterator<int32_t>;
/// Forward iterator for iterating over fixed64 values.
using const_fixed64_iterator = const_fixed_iterator<uint64_t>;
/// Forward iterator for iterating over sfixed64 values.
using const_sfixed64_iterator = const_fixed_iterator<int64_t>;
/// Forward iterator for iterating over float values.
using const_float_iterator = const_fixed_iterator<float>;
/// Forward iterator for iterating over double values.
using const_double_iterator = const_fixed_iterator<double>;
///@{
/**
* @name Repeated packed field accessor functions
......@@ -863,7 +881,7 @@ public:
* @pre The current field must be of type "repeated packed fixed32".
* @post The current field was consumed and there is no current field now.
*/
auto get_packed_fixed32() -> decltype(packed_fixed<uint32_t>()) {
iterator_range<pbf_reader::const_fixed32_iterator> get_packed_fixed32() {
return packed_fixed<uint32_t>();
}
......@@ -876,7 +894,7 @@ public:
* @pre The current field must be of type "repeated packed sfixed32".
* @post The current field was consumed and there is no current field now.
*/
auto get_packed_sfixed32() -> decltype(packed_fixed<int32_t>()) {
iterator_range<pbf_reader::const_sfixed32_iterator> get_packed_sfixed32() {
return packed_fixed<int32_t>();
}
......@@ -889,7 +907,7 @@ public:
* @pre The current field must be of type "repeated packed fixed64".
* @post The current field was consumed and there is no current field now.
*/
auto get_packed_fixed64() -> decltype(packed_fixed<uint64_t>()) {
iterator_range<pbf_reader::const_fixed64_iterator> get_packed_fixed64() {
return packed_fixed<uint64_t>();
}
......@@ -902,7 +920,7 @@ public:
* @pre The current field must be of type "repeated packed sfixed64".
* @post The current field was consumed and there is no current field now.
*/
auto get_packed_sfixed64() -> decltype(packed_fixed<int64_t>()) {
iterator_range<pbf_reader::const_sfixed64_iterator> get_packed_sfixed64() {
return packed_fixed<int64_t>();
}
......@@ -915,7 +933,7 @@ public:
* @pre The current field must be of type "repeated packed float".
* @post The current field was consumed and there is no current field now.
*/
auto get_packed_float() -> decltype(packed_fixed<float>()) {
iterator_range<pbf_reader::const_float_iterator> get_packed_float() {
return packed_fixed<float>();
}
......@@ -928,7 +946,7 @@ public:
* @pre The current field must be of type "repeated packed double".
* @post The current field was consumed and there is no current field now.
*/
auto get_packed_double() -> decltype(packed_fixed<double>()) {
iterator_range<pbf_reader::const_double_iterator> get_packed_double() {
return packed_fixed<double>();
}
......
......@@ -23,12 +23,12 @@ documentation.
#define PROTOZERO_VERSION_MINOR 6
/// The patch number
#define PROTOZERO_VERSION_PATCH 0
#define PROTOZERO_VERSION_PATCH 1
/// The complete version number
#define PROTOZERO_VERSION_CODE (PROTOZERO_VERSION_MAJOR * 10000 + PROTOZERO_VERSION_MINOR * 100 + PROTOZERO_VERSION_PATCH)
/// Version number as string
#define PROTOZERO_VERSION_STRING "1.6.0"
#define PROTOZERO_VERSION_STRING "1.6.1"
#endif // PROTOZERO_VERSION_HPP
......@@ -77,9 +77,9 @@ if(PROTOBUF_FOUND)
if(EXISTS "${_full_src_dir}/writer_test_cases.cpp")
message(STATUS " Adding ${_dir}")
set(_full_bin_dir "${CMAKE_CURRENT_BINARY_DIR}/t/${_dir}")
set(_proto_file "${_full_src_dir}/testcase.proto")
set(_src_file "${_full_bin_dir}/testcase.pb.cc")
set(_hdr_file "${_full_bin_dir}/testcase.pb.h")
set(_proto_file "${_full_src_dir}/${_dir}_testcase.proto")
set(_src_file "${_full_bin_dir}/${_dir}_testcase.pb.cc")
set(_hdr_file "${_full_bin_dir}/${_dir}_testcase.pb.h")
file(MAKE_DIRECTORY ${_full_bin_dir})
......
......@@ -281,11 +281,13 @@ TEST_CASE("write from different types of iterators: " PBF_TYPE_NAME) {
REQUIRE(it_range.front() == 1); it_range.drop_front();
REQUIRE(it_range.front() == 4); it_range.drop_front();
REQUIRE(std::distance(it_range.begin(), it_range.end()) == 3);
REQUIRE(it_range.size() == 3);
REQUIRE(it_range.front() == 9); it_range.drop_front();
REQUIRE(it_range.front() == 16); it_range.drop_front();
REQUIRE(it_range.front() == 25); it_range.drop_front();
REQUIRE(it_range.empty());
REQUIRE(std::distance(it_range.begin(), it_range.end()) == 0);
REQUIRE(it_range.size() == 0); // NOLINT clang-tidy: readability-container-size-empty
REQUIRE_THROWS_AS(it_range.front(), const assert_error&);
......
......@@ -3,7 +3,7 @@
#include <test.hpp> // IWYU pragma: keep
#include "t/bool/testcase.pb.h"
#include "t/bool/bool_testcase.pb.h"
TEST_CASE("write bool field and check with libprotobuf") {
......
#include <test.hpp>
#include "t/bytes/testcase.pb.h"
#include "t/bytes/bytes_testcase.pb.h"
TEST_CASE("write bytes field and check with libprotobuf") {
......
#include <test.hpp>
#include "t/double/testcase.pb.h"
#include "t/double/double_testcase.pb.h"
TEST_CASE("write double field and check with libprotobuf") {
......
......@@ -59,7 +59,7 @@ TEST_CASE("byte swap double") {
protozero::detail::byteswap_inplace(&a);
protozero::detail::byteswap_inplace(&a);
REQUIRE(1.1 == a);
REQUIRE(a == Approx(1.1));
}
TEST_CASE("byte swap float") {
......@@ -67,6 +67,6 @@ TEST_CASE("byte swap float") {
protozero::detail::byteswap_inplace(&a);
protozero::detail::byteswap_inplace(&a);
REQUIRE(1.1f == a);
REQUIRE(a == Approx(1.1f));
}
#include <test.hpp>
#include "t/enum/testcase.pb.h"
#include "t/enum/enum_testcase.pb.h"
TEST_CASE("write enum field and check with libprotobuf") {
......
#include <test.hpp>
#include "t/fixed32/testcase.pb.h"
#include "t/fixed32/fixed32_testcase.pb.h"
TEST_CASE("write fixed32 field and check with libprotobuf") {
......
#include <test.hpp>
#include "t/int32/testcase.pb.h"
#include "t/int32/int32_testcase.pb.h"
TEST_CASE("write int32 field and check with libprotobuf") {
......
#include <test.hpp>
#include "t/message/testcase.pb.h"
#include "t/message/message_testcase.pb.h"
TEST_CASE("write message field and check with libprotobuf") {
......
#include <test.hpp>
#include "t/nested/testcase.pb.h"
#include "t/nested/nested_testcase.pb.h"
TEST_CASE("write nested message fields and check with libprotobuf") {
......
#include <test.hpp>
#include "t/repeated/testcase.pb.h"
#include "t/repeated/repeated_testcase.pb.h"
TEST_CASE("write repeated fields and check with libprotobuf") {
......
......@@ -16,6 +16,7 @@ TEST_CASE("read repeated packed bool field: one") {
REQUIRE(item.next());
const auto it_range = item.get_packed_bool();
REQUIRE(std::distance(it_range.begin(), it_range.end()) == 1);
REQUIRE(it_range.size() == 1);
REQUIRE_FALSE(item.next());
......@@ -31,6 +32,7 @@ TEST_CASE("read repeated packed bool field: many") {
REQUIRE(item.next());
const auto it_range = item.get_packed_bool();
REQUIRE(std::distance(it_range.begin(), it_range.end()) == 4);
REQUIRE(it_range.size() == 4);
REQUIRE_FALSE(item.next());
......
......@@ -28,7 +28,7 @@ TEST_CASE("read repeated packed double field") {
const auto it_range = item.get_packed_double();
REQUIRE_FALSE(item.next());
REQUIRE(*it_range.begin() == 17.34);
REQUIRE(*it_range.begin() == Approx(17.34));
REQUIRE(std::next(it_range.begin()) == it_range.end());
}
......@@ -41,12 +41,39 @@ TEST_CASE("read repeated packed double field") {
REQUIRE_FALSE(item.next());
auto it = it_range.begin();
REQUIRE(*it++ == 17.34);
REQUIRE(*it++ == 0.0);
REQUIRE(*it++ == 1.0);
REQUIRE(*it++ == Approx(17.34));
REQUIRE(*it++ == Approx( 0.0));
REQUIRE(*it++ == Approx( 1.0));
REQUIRE(*it++ == std::numeric_limits<double>::min());
REQUIRE(*it++ == std::numeric_limits<double>::max());
REQUIRE(it == it_range.end());
it = it_range.begin();
auto it2 = it + 1;
REQUIRE(it2 > it);
REQUIRE(it < it2);
REQUIRE(it <= it_range.begin());
REQUIRE(it >= it_range.begin());
REQUIRE(*it2 == Approx(0.0));
auto it3 = 1 + it;
REQUIRE(*it3 == Approx(0.0));
auto it4 = it2 - 1;
REQUIRE(*it4 == Approx(17.34));
it4 += 2;
REQUIRE(*it4 == Approx(1.0));
it4 -= 2;
REQUIRE(*it4 == Approx(17.34));
it4 += 2;
REQUIRE(*it4 == Approx(1.0));
REQUIRE(*it4-- == Approx(1.0));
REQUIRE(*it4 == Approx(0.0));
REQUIRE(*--it4 == Approx(17.34));
REQUIRE(it4[0] == Approx(17.34));
REQUIRE(it4[1] == Approx(0.0));
REQUIRE(std::distance(it_range.begin(), it_range.end()) == 5);
REQUIRE(it_range.end() - it_range.begin() == 5);
REQUIRE(it_range.begin() - it_range.end() == -5);
}
SECTION("end_of_buffer") {
......
#include <test.hpp>
#include "t/repeated_packed_fixed32/testcase.pb.h"
#include "t/repeated_packed_fixed32/repeated_packed_fixed32_testcase.pb.h"
TEST_CASE("write repeated packed fixed32 field and check with libprotobuf") {
......
......@@ -28,7 +28,7 @@ TEST_CASE("read repeated packed float field") {
auto it_range = item.get_packed_float();
REQUIRE_FALSE(item.next());
REQUIRE(*it_range.begin() == 17.34f);
REQUIRE(*it_range.begin() == Approx(17.34f));
REQUIRE(std::next(it_range.begin()) == it_range.end());
}
......@@ -41,9 +41,9 @@ TEST_CASE("read repeated packed float field") {
REQUIRE_FALSE(item.next());
auto it = it_range.begin();
REQUIRE(*it++ == 17.34f);
REQUIRE(*it++ == 0.0f);
REQUIRE(*it++ == 1.0f);
REQUIRE(*it++ == Approx(17.34f));
REQUIRE(*it++ == Approx( 0.0f));
REQUIRE(*it++ == Approx( 1.0f));
REQUIRE(*it++ == std::numeric_limits<float>::min());
REQUIRE(*it++ == std::numeric_limits<float>::max());
REQUIRE(it == it_range.end());
......
#include <test.hpp>
#include "t/string/testcase.pb.h"
#include "t/string/string_testcase.pb.h"
TEST_CASE("write string field and check with libprotobuf") {
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment