Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ARROW-4314: [Rust] Strongly-typed reading of Parquet data #3461

Closed
wants to merge 40 commits into from
Closed

ARROW-4314: [Rust] Strongly-typed reading of Parquet data #3461

wants to merge 40 commits into from

Commits on Aug 23, 2019

  1. Enable typed reading of records

    @alecmocatta
    alecmocatta committed Aug 23, 2019
    Configuration menu
    Copy the full SHA
    588f167 View commit details
    Browse the repository at this point in the history

  2. Make Value::as_* take &self; Add Value::into_*

    @alecmocatta
    alecmocatta committed Aug 23, 2019
    Configuration menu
    Copy the full SHA
    e4aad2c View commit details
    Browse the repository at this point in the history

  3. Implement Date, Time, Decimal, Bson, Json, Enum

    @alecmocatta
    alecmocatta committed Aug 23, 2019
    Configuration menu
    Copy the full SHA
    2a90f59 View commit details
    Browse the repository at this point in the history

  4. Avoid stack overflow by boxing RleDecoder's array

    @alecmocatta
    alecmocatta committed Aug 23, 2019
    Configuration menu
    Copy the full SHA
    0333d6b View commit details
    Browse the repository at this point in the history

  5. Add license headers

    @alecmocatta
    alecmocatta committed Aug 23, 2019
    Configuration menu
    Copy the full SHA
    4556a10 View commit details
    Browse the repository at this point in the history

  6. Initial derive macro implementation

    @alecmocatta
    alecmocatta committed Aug 23, 2019
    Configuration menu
    Copy the full SHA
    8b101c4 View commit details
    Browse the repository at this point in the history

  7. Fix derive macro on structs with type parameters

    @alecmocatta
    alecmocatta committed Aug 23, 2019
    Configuration menu
    Copy the full SHA
    6d7323b View commit details
    Browse the repository at this point in the history

  8. Remove unnecessary state from readers; Remove Root from Deserialize b… 

    …ounds so can deserialize foreign structs; Test #[derive(Deserialize)]; Better honour TripletIter API
    @alecmocatta
    alecmocatta committed Aug 23, 2019
    Configuration menu
    Copy the full SHA
    71ffc48 View commit details
    Browse the repository at this point in the history

  9. Add benchmarks from original repo

    @alecmocatta
    alecmocatta committed Aug 23, 2019
    Configuration menu
    Copy the full SHA
    1565a56 View commit details
    Browse the repository at this point in the history

  10. Enable existential types; From and PartialEq for Value

    @alecmocatta
    alecmocatta committed Aug 23, 2019
    Configuration menu
    Copy the full SHA
    30a24bc View commit details
    Browse the repository at this point in the history

  11. Avoid Boxing all Value::Options

    @alecmocatta
    alecmocatta committed Aug 23, 2019
    Configuration menu
    Copy the full SHA
    7b9e5f6 View commit details
    Browse the repository at this point in the history

  12. Remove benchmarks, to be re-added in a separate PR

    @alecmocatta
    alecmocatta committed Aug 23, 2019
    Configuration menu
    Copy the full SHA
    23e3af7 View commit details
    Browse the repository at this point in the history

  13. Move Deserialize into record; minor cleanup

    @alecmocatta
    alecmocatta committed Aug 23, 2019
    Configuration menu
    Copy the full SHA
    a66858e View commit details
    Browse the repository at this point in the history

  14. Add projection tests using derive macro

    @alecmocatta
    alecmocatta committed Aug 23, 2019
    Configuration menu
    Copy the full SHA
    d758139 View commit details
    Browse the repository at this point in the history

  15. Better printing of schema

    @alecmocatta
    alecmocatta committed Aug 23, 2019
    Configuration menu
    Copy the full SHA
    421e946 View commit details
    Browse the repository at this point in the history

  16. Reflect upstream rustfmt config change

    @alecmocatta
    alecmocatta committed Aug 23, 2019
    Configuration menu
    Copy the full SHA
    38a0023 View commit details
    Browse the repository at this point in the history

  17. Test parsing and printing of schemas

    @alecmocatta
    alecmocatta committed Aug 23, 2019
    Configuration menu
    Copy the full SHA
    ec59b24 View commit details
    Browse the repository at this point in the history

Commits on Aug 24, 2019

  1. Clean up warnings

    @alecmocatta
    alecmocatta committed Aug 24, 2019
    Configuration menu
    Copy the full SHA
    4270c9b View commit details
    Browse the repository at this point in the history

  2. Avoid resizing Group's Vec during read

    @alecmocatta
    alecmocatta committed Aug 24, 2019
    Configuration menu
    Copy the full SHA
    5c3bb68 View commit details
    Browse the repository at this point in the history

  3. Update time tests

    @alecmocatta
    alecmocatta committed Aug 24, 2019
    Configuration menu
    Copy the full SHA
    0fc3db9 View commit details
    Browse the repository at this point in the history

  4. Unbox result of FileReader::get_row_group so one can call get_row_ite… 

    …r<T> on it
    @alecmocatta
    alecmocatta committed Aug 24, 2019
    Configuration menu
    Copy the full SHA
    757f3ed View commit details
    Browse the repository at this point in the history

  5. Impl Default for Schemas where sensical; clean up interface

    @alecmocatta
    alecmocatta committed Aug 24, 2019
    Configuration menu
    Copy the full SHA
    31fb201 View commit details
    Browse the repository at this point in the history

  6. List Rust <-> Parquet type correspondence

    @alecmocatta
    alecmocatta committed Aug 24, 2019
    Configuration menu
    Copy the full SHA
    6027368 View commit details
    Browse the repository at this point in the history

  7. Avoid need for box_syntax feature

    @alecmocatta
    alecmocatta committed Aug 24, 2019
    Configuration menu
    Copy the full SHA
    50380c9 View commit details
    Browse the repository at this point in the history

  8. Revert "Fix some UB; make clippy less unhappy; cleanup"

    @alecmocatta
    alecmocatta committed Aug 24, 2019
    Configuration menu
    Copy the full SHA
    b45e3c4 View commit details
    Browse the repository at this point in the history

  9. Document various components

    @alecmocatta
    alecmocatta committed Aug 24, 2019
    Configuration menu
    Copy the full SHA
    a9dedef View commit details
    Browse the repository at this point in the history

  10. Document more and add invalid map test

    @alecmocatta
    alecmocatta committed Aug 24, 2019
    Configuration menu
    Copy the full SHA
    c2a6f55 View commit details
    Browse the repository at this point in the history

  11. More permissive with unknown LogicalTypes, Faster [u8; N]; Docs

    @alecmocatta
    alecmocatta committed Aug 24, 2019
    Configuration menu
    Copy the full SHA
    08b4bf4 View commit details
    Browse the repository at this point in the history

  12. Remove invalid map derived test 

    The type system prevents this, and the untyped test is invalid_map_type in record/schemas.rs
    @alecmocatta
    alecmocatta committed Aug 24, 2019
    Configuration menu
    Copy the full SHA
    a8c339d View commit details
    Browse the repository at this point in the history

  13. Return errors thru iterator

    @alecmocatta
    alecmocatta committed Aug 24, 2019
    Configuration menu
    Copy the full SHA
    5c41457 View commit details
    Browse the repository at this point in the history

  14. Make RowIter own rather than borrow FileReader

    @alecmocatta
    alecmocatta committed Aug 24, 2019
    Configuration menu
    Copy the full SHA
    e64ba9c View commit details
    Browse the repository at this point in the history

  15. Change derive const from DESERIALIZE -> RECORD

    @alecmocatta
    alecmocatta committed Aug 24, 2019
    Configuration menu
    Copy the full SHA
    0b727a2 View commit details
    Browse the repository at this point in the history

  16. Make derive macro more hygienic

    @alecmocatta
    alecmocatta committed Aug 24, 2019
    Configuration menu
    Copy the full SHA
    5b969fd View commit details
    Browse the repository at this point in the history

  17. Implement From/Into<chrono::DateTime<Utc>> for Timestamp

    @alecmocatta
    alecmocatta committed Aug 24, 2019
    Configuration menu
    Copy the full SHA
    0ee0a8b View commit details
    Browse the repository at this point in the history

  18. Use a faster hashmap for Group, and wrap it in Arc rather than Rc so … 

    …that Group, Value etc are Send; Return Err on overflowing Timetamp math
    @alecmocatta
    alecmocatta committed Aug 24, 2019
    Configuration menu
    Copy the full SHA
    4ce5c0e View commit details
    Browse the repository at this point in the history

  19. Don't rely on existential types; Docs now build; Doc improvements

    @alecmocatta
    alecmocatta committed Aug 24, 2019
    Configuration menu
    Copy the full SHA
    b36e0ed View commit details
    Browse the repository at this point in the history

  20. Remove #[derive(Record)] for another commit

    @alecmocatta
    alecmocatta committed Aug 24, 2019
    Configuration menu
    Copy the full SHA
    3e1f368 View commit details
    Browse the repository at this point in the history

  21. Integrate @sunchao's feedback

    @alecmocatta
    alecmocatta committed Aug 24, 2019
    Configuration menu
    Copy the full SHA
    49259dd View commit details
    Browse the repository at this point in the history

  22. Remove get_row_iter argument; document sound usage of unsafe

    @alecmocatta
    alecmocatta committed Aug 24, 2019
    Configuration menu
    Copy the full SHA
    e0c4064 View commit details
    Browse the repository at this point in the history

  23. fix tests

    @alecmocatta
    alecmocatta committed Aug 24, 2019
    Configuration menu
    Copy the full SHA
    c0b5e45 View commit details
    Browse the repository at this point in the history