- Sparse tensor support in TFF:
- EW - We have Keras models that we want to port to TFF, they have sparse
tensors
- Simply mapping to dense tensors results in unacceptable memory cost and slowness in our use case, so we’re looking to avoid that
- ZG on existing sparse tensor support in TFF
- Issues mentioned on GitHub mostly related to tf.data.Dataset
- Mostly works otherwise, but it requires some DIY, particularly w.r.t, aggregations where we can’t just naively do a sparse sum on the triple of constituent tensors, that wouldn’t have the desired outcome
- (question about relative importance)
- EW - this is not blocking for us, but a good performance/resoruce footprint optimization
- ZG - with respect to the GitHub issues, might work around by hiding dataset inside the TFF computation, so it’s not part of the input-output boundary
- KO - clarifying that our “it mostly works” comment refers to the common
practice of representing/handling sparse tensors as tuples of dense
tensors. Have you tried dealing with sparse as tuples of dense tensors
for datasets usage as well?
- EW - haven’t tried yet
- KO - sparse in this conversation has come up in two places - for model
parameters, but also for sparse input data - are both equally important?
- EW - would ideally have both
- KO - one action item for Ewan to try to work with tuples of dense tensors that represent the constituent parts.
- KO - this still leaves a question about better APIs/helpers for sparse tensor handling, but can unblock this particular use case. Thoughts on the API?
- EW - ideally could this just be transparent (no need to do anything
special for sparse by the customer using TFF and it just works)
- KO, ZG - in some cases, it’s not obvious, e.g., for aggregation - there’s potentially more than one way to aggregate the constituent parts of sparse tensors, a choice ideally to be made by the customer
- KR - probably having a small family of dedicated “sparse sum” symbols is most actionable
- KO - perhaps we can start by prototyping the version of sparse sum needed by EW and upstream it to TFF as a generic sparse sum operator to seed this, and build on that (to follow up on this offline - maybe on discord)
- EW +1
- EW - We have Keras models that we want to port to TFF, they have sparse
tensors
- Jeremy’s proposal, continuing from 2 weeks ago:
- TFF Tech Note: Client initiated connections
- (todo for all to review it later as it was just shared shortly before the meeting)
- (Jeremy is presenting)
- JL - proposing the “task store” abstraction for exchanging requests between a “Cloud” and the per-client executors (e.g., in browsers), with the latter pulling tasks from a centralized “task store”. Has something like this been considered in any other context?
- KR - yes, in failure handling scenarios
- More hairly problems, though - state transfer across executors is difficult, not sure how much carries over to the scenario presented by Jeremy
- HV - can the executors in the leaves be stateless
- JL - this would make it more like the SysML paper on cross-device
- (question about performance in this scenario, compared to bi-directional streaming in a way that more closely resembles the native TFF protocol)
- JL - ack that there are latency considerations
- bi-directional streaming not supported in some transports, so not always a viable option
- (ran out of time)
- (to be continued in 2 weeks - first point of agenda for the next meeting, Jeremy will join)
Except as otherwise noted, the content of this page is licensed under the Creative Commons Attribution 4.0 License, and code samples are licensed under the Apache 2.0 License. For details, see the Google Developers Site Policies. Java is a registered trademark of Oracle and/or its affiliates.
Last updated 2022-12-06 UTC.
[[["Easy to understand","easyToUnderstand","thumb-up"],["Solved my problem","solvedMyProblem","thumb-up"],["Other","otherUp","thumb-up"]],[["Missing the information I need","missingTheInformationINeed","thumb-down"],["Too complicated / too many steps","tooComplicatedTooManySteps","thumb-down"],["Out of date","outOfDate","thumb-down"],["Samples / code issue","samplesCodeIssue","thumb-down"],["Other","otherDown","thumb-down"]],["Last updated 2022-12-06 UTC."],[],[]]