in scripts/eval_script.py there is a data type check:
if strategy_name == "efficient_kascade" and model.config.dtype != torch.float16:
When running Meta-Llama-3.1-8B-Instruct it throws error that LlamaConfig does not have dtype member. This class has torch_dtype though. Probably the type check should be
if strategy_name == "efficient_kascade" and model.config.torch_dtype != torch.float16:
Although I'm not sure if it will also work for other model config classes.