# Generate Mock but realistic data using NLP
**URL:**
**Author:**  [@rajasingh012](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Frajasingh012)%5C
**Post date:** [September 29, 2024, 4:53am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fgenerate-mock-but-realistic-data-using-nlp%2F109254%2F1 "2024-09-29T04:53:52Z")
I want to generate realistic test data. It should support customizable Fields, structure of data by specifying field names and types. It should support a wide range of data types, including names, addresses, email addresses, electrical products, household products etc.
I plan to use small language model or prefix based llm, because i don’t want full sentence, just word(s). model response is json format. I want to implement similar to [https://www.mockaroo.com/](https://www.mockaroo.com/) by using AI
The main purpose of the model is to replace data in production database.
---
**Author:**  [@Pankaj8922](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FPankaj8922)%5C
**Post date:** [September 29, 2024, 7:01am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fgenerate-mock-but-realistic-data-using-nlp%2F109254%2F2 "2024-09-29T07:01:24Z")
I can help you with that, but you gotta manage and The Resource part like GPU, and you will have to give me first 5k samples of kind of data you want to genrate.
---
**Author:**  [@John6666](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FJohn6666)%5C
**Post date:** [September 29, 2024, 7:02am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fgenerate-mock-but-realistic-data-using-nlp%2F109254%2F3 "2024-09-29T07:02:28Z")
I play a lot with LLMs, and while they are creative, they are not very good at strict formatting output, etc.
So, what about using Function Calling, which I have only used a few times.
If you want LLMs to manipulate routine data, you can expect more than normal performance.
> **[GitHub - imaurer/awesome-llm-json: Resource list for generating JSON using LLMs via...](https://github.com/imaurer/awesome-llm-json)**
>
> Resource list for generating JSON using LLMs via function calling, tools, CFG. Libraries, Models, Notebooks, etc.
For specific code, etc., it would be quicker to find and imitate Space for similar uses in Spaces.
> **[Spaces - Hugging Face](https://huggingface.co/spaces?sort=trending&search=json)**
>
> Discover amazing ML apps made by the community
---
**Author:**  [@Pankaj8922](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FPankaj8922)%5C
**Post date:** [September 29, 2024, 7:05am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fgenerate-mock-but-realistic-data-using-nlp%2F109254%2F4 "2024-09-29T07:05:29Z")
I have created a Tamplate based Generation technique which can help in it but first we will need to fine tune model in a specific way. With totally diverse data. Then it can be used to generate High quality data and as much as want. We can use llms between 0.5B to 1.5B. for genration.
---
**Author:**  [@rajasingh012](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Frajasingh012)%5C
**Post date:** [September 29, 2024, 1:53pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fgenerate-mock-but-realistic-data-using-nlp%2F109254%2F5 "2024-09-29T13:53:01Z")
Hard part is gathering totally diverse data and cleaning it. I plan to use existing model like Mistral or llama. Does it work
---
**Author:**  [@system](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsystem)%5C
**Post date:** [October 1, 2024, 1:30am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fgenerate-mock-but-realistic-data-using-nlp%2F109254%2F6 "2024-10-01T01:30:28Z")
This topic was automatically closed 12 hours after the last reply. New replies are no longer allowed.