Huggingface is a prominent open-source platform for machine learning and natural language processing developers and researchers. It provides resources like models, datasets, etc. for application and research. The transformer library in Huggingface is powerful for natural language processing tasks. It enables users to import and use pretrained transformer models easily.
When we call the transformer using this " model = AutoModelForCausalLM.from_pretrained("bert-base-uncased")" method, It will automatically load the model into the CPU. We need to call 'Cuda' for loading the model into the GPU.
Solution:
Huggingface acceleration could help move the model to GPU before it's fully loaded in the CPU, so it worked when
GPU memory > model size > CPU memory
by usingdevice_map = 'cuda'
!pip install accelerate
then use
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("bert-base-uncased", device_map = 'cuda')Thank you for reading the article.
