Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

reuse inner conv #2

Open
matpalm opened this issue Nov 12, 2023 · 0 comments
Open

reuse inner conv #2

matpalm opened this issue Nov 12, 2023 · 0 comments

Comments

@matpalm
Copy link
Owner

matpalm commented Nov 12, 2023

the v1.0 release code is clumsy and even though we have cycles left we can't add another conv.

but we can reuse one using tied weights

so we instead of training input -> conv0 -> output we could train ``input -> conv0 -> conv0 -> output` by just reusing qconv layer which will train with tied weights, then at inference time we just need 1 additional activation cache and the ability to switch in/out for the inner cache ( which i guess is done with registering? )

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant