Metadata-Version: 2.1
Name: bitlinear-pytorch
Version: 0.0.2
Summary: Implementation of the BitLinear layer from: The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits
Home-page: https://github.com/ingur/bitlinear-pytorch
Author: Ingur Veken
Author-email: Ingur Veken <ingurv99@gmail.com>, Niels Rouws <nrouws@gmail.com>
License: MIT License
        
        Copyright (c) 2024 Ingur Veken
        
        Permission is hereby granted, free of charge, to any person obtaining a copy
        of this software and associated documentation files (the "Software"), to deal
        in the Software without restriction, including without limitation the rights
        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
        copies of the Software, and to permit persons to whom the Software is
        furnished to do so, subject to the following conditions:
        
        The above copyright notice and this permission notice shall be included in all
        copies or substantial portions of the Software.
        
        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
        SOFTWARE.
        
Project-URL: Homepage, https://github.com/ingur/bitlinear-pytorch
Project-URL: Issues, https://github.com/ingur/bitlinear-pytorch/issues
Keywords: pytorch,quantization,bitlinear,linear,layer,ternary,binary,quantized,quantized weights
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: License :: OSI Approved :: MIT License
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: torch >=1.10.0
Provides-Extra: test
Requires-Dist: pytest ; extra == 'test'
Requires-Dist: pytest-runner ; extra == 'test'

# bitlinear-pytorch

Implementation of the BitLinear layer from: [The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits](https://arxiv.org/abs/2402.17764)

## Install
```bash
pip install bitlinear_pytorch
```

## Usage
```python
import torch
from bitlinear_pytorch import BitLinear, replace_linear_with_bitlinear

class TinyMLP(nn.Module):
    def __init__(self):
        super(TinyMLP, self).__init__()

        self.layers = nn.Sequential(
            nn.Linear(784, 256),
            nn.ReLU(),
            nn.Linear(256, 128),
            nn.ReLU(),
            nn.Linear(128, 10),
        )

    def forward(self, x):
        return self.layers(x)

model = TinyMLP()
replace_linear_with_bitlinear(model)

# or use BitLinear directly
bitlinear = BitLinear(784, 256)
```

## License
MIT

## Citation
```bibtex
@misc{ma2024era,
      title={The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits}, 
      author={Shuming Ma and Hongyu Wang and Lingxiao Ma and Lei Wang and Wenhui Wang and Shaohan Huang and Li Dong and Ruiping Wang and Jilong Xue and Furu Wei},
      year={2024},
      eprint={2402.17764},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}
```

## TODO
- [x] Implement base BitLinear layer
- [x] Add example usage
- [ ] Implement memory efficient weight encoding/decoding
- [ ] Implement Fast Inference (CUDA/CPU/VHDL)
