From the model card
Fields below are copied from the tags and counters on the HuggingFace repository tiiuae/falcon-7b at our last fetch. They are set by the uploader, not verified by us; rows with no tag are omitted. How this page is made.
- Publisher (HF namespace)
- tiiuae
- Pipeline tag
- text-generation
- Library
- Transformers
- Framework tags
- PyTorch
- Weight formats
- safetensors
- License tag
apache-2.0— read the license file in the repo before relying on it- Language tags
- English (en)
- Papers cited
- arXiv:2205.14135, arXiv:1911.02150, arXiv:2101.00027, arXiv:2005.14165, arXiv:2104.09864, arXiv:2306.01116
- Datasets declared
- tiiuae/falcon-refinedweb
- Downloads (HF counter at last fetch)
- 495,588
- Likes (HF counter at last fetch)
- 1,105
- Model card
- https://huggingface.co/tiiuae/falcon-7b
Use cases
- Historical benchmark reproduction for 2023-era open 7B LLMs
- Fine-tuning starting point on RefinedWeb-style web text distributions
- Text continuation generation for applications suited to base model prompting
- Teaching LLM training concepts with a documented, reproducible checkpoint
- Comparative study of web-crawl data filtering methods and their downstream impact
Pros
- RefinedWeb training data is extensively documented with filtering pipeline details
- Apache 2.0 license; no restrictions on commercial use
- 1102 likes; historically significant and heavily referenced in open LLM research
- Multi-query attention variant improves inference efficiency vs standard MHA
Cons
- Superseded by Falcon 2, Llama 2/3, Mistral, and Qwen across all standard benchmarks
- Base model; instruction following requires fine-tuning
- Custom model code needed for some inference configurations
- RefinedWeb-focused training produces weaker code and math performance vs specialised models
Tags
transformerspytorchsafetensorsfalcontext-generationcustom_codeendataset:tiiuae/falcon-refinedwebarxiv:2205.14135arxiv:1911.02150arxiv:2101.00027arxiv:2005.14165arxiv:2104.09864arxiv:2306.01116license:apache-2.0text-generation-inferenceregion:usdeploy:sagemakerdeploy:azure