Skip to content

Uses torch.fx to parallelize and transform pipelined models#193

Closed
michaelbenayoun wants to merge 33 commits intomainfrom
fx_transformations
Closed

Uses torch.fx to parallelize and transform pipelined models#193
michaelbenayoun wants to merge 33 commits intomainfrom
fx_transformations

Conversation

@michaelbenayoun
Copy link
Copy Markdown
Member

@michaelbenayoun michaelbenayoun commented Oct 19, 2022

What does this PR do?

  • Provides a custom tracer, built upon transformers.utils.fx.HFTracer allowing the trace and transform the models we support in optimum-graphcore
  • A set of pipelining related transformations
  • Rewrites all the parallelize / deparallelize methods we support
  • Integrates that with the IPUTrainer

@HuggingFaceDocBuilderDev
Copy link
Copy Markdown

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.

@michaelbenayoun michaelbenayoun changed the title Fx transformations Uses torch.fx to parallelize and transform pipelined models Oct 20, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants