# Increase Output length of Falcon 7b instruct

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fincrease-output-length-of-falcon-7b-instruct%2F45430%3E%5C
**Category:** Beginners\
**Created:** [July 3, 2023, 9:01am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fincrease-output-length-of-falcon-7b-instruct%2F45430 "2023-07-03T09:01:03Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Arisil](https://avatars.discourse-cdn.com/v4/letter/a/47e85d/32.png) [@Arisil](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FArisil)%5C
**Post date:** [July 3, 2023, 9:01am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fincrease-output-length-of-falcon-7b-instruct%2F45430%2F1 "2023-07-03T09:01:03Z")

</div>

Hello everyone,

I have made a deployment of Falcon 7B instruct using the AWS sagemaker SDK code in the model’s page.

```auto
import json
import sagemaker
import boto3
from sagemaker.huggingface import HuggingFaceModel, get_huggingface_llm_image_uri

try:
	role = sagemaker.get_execution_role()
except ValueError:
	iam = boto3.client('iam')
	role = iam.get_role(RoleName='sagemaker_execution_role')['Role']['Arn']

# Hub Model configuration. https://huggingface.co/models
hub = {
	'HF_MODEL_ID':'tiiuae/falcon-7b-instruct',
	'SM_NUM_GPUS': json.dumps(1)
}

# create Hugging Face Model Class
huggingface_model = HuggingFaceModel(
	image_uri=get_huggingface_llm_image_uri("huggingface",version="0.8.2"),
	env=hub,
	role=role, 
)

# deploy model to SageMaker Inference
predictor = huggingface_model.deploy(
	initial_instance_count=1,
	instance_type="ml.g5.2xlarge",
	container_startup_health_check_timeout=300,
  )
  
# send request
predictor.predict({
	"inputs": "Hey Falcon! Any recommendations for my holidays in Abu Dhabi?",
})

```

It is working just fine but the outputs are too short (just a couple of sentences). Where should the changes be made so that it managed to produce at least one paragraph ? Thank you in advance.

---

<div class="post-metadata">

**Author:** ![nielsr](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/nielsr/32/39617_2.png) [@nielsr](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fnielsr)%5C
**Post date:** [July 3, 2023, 9:57am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fincrease-output-length-of-falcon-7b-instruct%2F45430%2F2 "2023-07-03T09:57:28Z")

</div>

cc @philschmid

---

<div class="post-metadata">

**Author:** ![Arisil](https://avatars.discourse-cdn.com/v4/letter/a/47e85d/32.png) [@Arisil](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FArisil)%5C
**Post date:** [July 4, 2023, 9:55am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fincrease-output-length-of-falcon-7b-instruct%2F45430%2F3 "2023-07-04T09:55:29Z")

</div>

I had made some changes on the deployment section but afterwards I made another deployment changing the configuration on the hub

```auto
# Hub Model configuration. https://huggingface.co/models
hub = {
	'HF_MODEL_ID':'tiiuae/falcon-7b-instruct',
	'SM_NUM_GPUS': json.dumps(1),
    'MAX_TOTAL_TOKENS': json.dumps(2048),
}

```

And that seems to have increased the output
