Skip to content

update readme #5

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
sacrex wants to merge 1 commit into
base: main
Choose a base branch
from
Open

update readme #5

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# collaborative-inference
近年来，由于需要解决的实际问题日益复杂，使得深度学习模型越来越庞大，然而算力支撑开始捉襟见肘。因此，在资源不足、尤其是单机算力受限的情况下，部署深度学习模型进行推理面临着挑战。如果对深度学习模型的并行部分独立打包发送到不同的机器上进行推理，在不考虑传输的前提下，势必会减小单机压力，加快推理速度。而Ray作为一个高性能分布式任务执行框架，我们将并行模块打包注册为Ray的远程任务，可以利用Ray的架构并行执行该部分的推理，所以我们考虑以Ray的编程模式自动打包并重写深度学习并行部分的源码，利用Ray的分布式调度器和分布式对象存储器突破单机算力限制，实现推理加速。

如何基于模型并行的策略，加速深度学习模型推理是近来备受关注的问题。针对含有并行网络结构的复杂深度学习模型，我们开发了一套深度学习模型源码重构软件。该软件可以提取深度学习模型计算图结构，据此对给定的深度学习模型源码进行拆分重构，并将能够并行的部分打入Ray的标签，提交到Ray集群中进行并行推理。
如何基于模型并行的策略，加速深度学习模型推理是近来备受关注的问题。针对含有并行网络结构的复杂深度学习模型，我们开发了一套深度学习模型源码重构软件。该软件可以提取深度学习模型计算图结构，据此对给定的深度学习模型源码进行拆分重构，并将能够并行的部分打入Ray的标签，提交到分布式计算的Ray集群中进行并行推理。