Assessing the impact of AI-generated product names on e-commerce performance
DOI:
https://doi.org/10.20535/SRIT.2308-8893.2025.1.10Keywords:
large language models, AI-detection, e-commerce, product performanceAbstract
This paper studies the impact of Large Language Model (LLM) technology on the e-commerce industry. This work conducts a detailed review of the current implementation level of LLM technologies in the e-commerce industry. Next, it analyzes the approaches to detecting AI-generated text and determines the limitations of their application. The proposed methodology defines the impact of LLM models on the e-commerce industry based on a comparative analysis between indicators of machine-generated texts and e-commerce product metrics. Applying this methodology to real data, one of the most relevant data collected after the release of ChatGPT, the results of statistical analyses show a positive correlation between the studied indicators. It is proved that this dependence is dynamic and changes over time. The obtained implicit indicators measure the influence of LLM technologies on the e-commerce domain. This influence is expected to grow, requiring further research.
References
W.X. Zhao et al., “A survey of large language models,” ArXiv, 2023. doi: https://doi.org/10.48550/arXiv.2303.18223
A. Miaschi, D. Brunato, F. Dell’Orletta, and G. Venturi, “What makes my model perplexed? A linguistic investigation on neural language models perplexity,” in Proceedings of Deep Learning Inside Out (DeeLIO): The 2nd Workshop on Knowledge Extraction and Integration for Deep Learning Architectures, pp. 40–47, 2021. doi: 10.18653/v1/2021.deelio-1.5
D. Klakow, J. Peters, “Testing the correlation of word error rate and perplexity,” Speech Communication, vol. 38, no. 1–2, pp. 19–28, 2002. doi: 10.1016/S0167-6393(01)00041-3
S.F. Chen, D. Beeferman, and R. Rosenfeld, Evaluation metrics for language models. 1998. Available: https://kilthub.cmu.edu/articles/Evaluation_Metrics_For_Language_Models/6605324/files/12095765.pdf
A. Hans et al., “Spotting LLMs with binoculars: Zero-shot detection of machine-generated text,” ArXiv, 2024. doi: https://doi.org/10.48550/arXiv.2401.12070
A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, and I. Sutskever, “Language models are unsupervised multitask learners,” OpenAI blog, vol. 1, no. 8, p. 9, 2019. Available: https://insightcivic.s3.us-east-1.amazonaws.com/language-models.pdf
T.B. Brown, “Language models are few-shot learners,” ArXiv, 2020. doi: https://doi.org/10.48550/arXiv.2005.14165
D. Adiwardana et al., “Towards a human-like open-domain chatbot,” ArXiv, 2020. doi: https://doi.org/10.48550/arXiv.2001.09977
P. Lewis et al., “Retrieval-augmented generation for knowledge-intensive NLP tasks,” Advances in Neural Information Processing Systems, vol. 33, pp. 9459–9474, 2020. Available: https://proceedings.neurips.cc/paper/2020/hash/6b493230205f780e1bc26945df7481e5-Abstract.html
B. Miranda, A. Lee, S. Sundar, A. Casasola, and S. Koyejo, “Beyond Scale: The Diversity Coefficient as a Data Quality Metric for Variability in Natural Language Data,” ArXiv, 2024. doi: https://doi.org/10.48550/arXiv.2306.13840
Gemini Team Google, “Gemini: a family of highly capable multimodal models,” ArXiv, 2023. doi: https://doi.org/10.48550/arXiv.2312.11805
H. Touvron et al., “Llama 2: Open foundation and fine-tuned chat models,” ArXiv, 2023. doi: https://doi.org/10.48550/arXiv.2307.09288
S.E. Spatharioti, D.M. Rothschild, D.G. Goldstein, and J.M. Hofman, “Comparing traditional and LLM-based search for consumer choice: A randomized experiment,” ArXiv, 2023. doi: https://doi.org/10.48550/arXiv.2307.03744
S. Frieder, J. Berner, P. Petersen, and T. Lukasiewicz, “Large language models for mathematicians,” ArXiv, 2023. doi: https://doi.org/10.48550/arXiv.2312.04556
F. Zeng, W. Gan, Y. Wang, N. Liu, and P.S. Yu, “Large language models for robotics: A survey,” ArXiv, 2023. doi: https://doi.org/10.48550/arXiv.2311.07226
G. Sousa, “Natural Language Processing and its applications in e-business,” Cadernos de Investigação do Mestrado em Negócio Eletrónico, vol. 2, 2022. doi: https://doi.org/10.56002/ceos.0070_cimne_1_2
Y. Huang, “Research on the Application of Natural Language Processing Technology in E-commerce,” in ISCTT 2021; 6th International Conference on Information Science, Computer Technology and Transportation, 2021, pp. 1–5. Available: https://ieeexplore.ieee.org/abstract/document/9738909
M. Chen, Q. Tang, S. Wiseman, and K. Gimpel, “Controllable paraphrase generation with a syntactic exemplar,” ArXiv, 2019. doi: https://doi.org/10.48550/arXiv.1906.00565
Q. Chen, J. Lin, Y. Zhang, H. Yang, J. Zhou, and J. Tang, “Towards knowledge-based personalized product description generation in e-commerce,” in Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 2019, pp. 3040–3050. doi: 10.1145/3292500.333072
Q. Ren et al., “A survey on fairness of large language models in e-commerce: progress, application, and challenge,” ArXiv, 2024. doi: https://doi.org/10.48550/arXiv.2405.13025
J. Zhou, B. Liu, J.N.A.Y. Hong, K.-C. Lee, and M. Wen, “Leveraging Large Language Models for Enhanced Product Descriptions in eCommerce,” ArXiv, 2023. doi: https://doi.org/10.48550/arXiv.2310.18357
K.I. Roumeliotis, N.D. Tselikas, and D.K. Nasiopoulos, “LLMs in e-commerce: a comparative analysis of GPT and LLaMA models in product review evaluation,” Natural Language Processing Journal, vol. 6, p. 100056, 2024. doi: 10.1016/j.nlp.2024.100056
G. Chodak, K. Błażyczek, “Large Language Models for Search Engine Optimization in E-commerce,” in International Advanced Computing Conference, pp. 333–344, 2023. doi: 10.1007/978-3-031-56700-1_27
B. Peng, X. Ling, Z. Chen, H. Sun, and X. Ning, “eCeLLM: Generalizing Large Language Models for E-commerce from Large-scale, High-quality Instruction Data,” ArXiv, 2024. doi: https://doi.org/10.48550/arXiv.2402.08831
C. Herold, M. Kozielski, L. Ekimov, P. Petrushkov, P.-Y. Vandenbussche, and S. Khadivi, “LiLiuM: eBay’s Large Language Models for e-commerce,” ArXiv, 2024. doi: https://doi.org/10.48550/arXiv.2406.12023
A. Chowdhery et al., “Palm: Scaling language modeling with pathways,” Journal of Machine Learning Research, vol. 24, no. 240, pp. 1–113, 2023. Available: http://jmlr.org/papers/v24/22-1144.html
K. Krishna, Y. Song, M. Karpinska, J. Wieting, and M. Iyyer, “Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense,” Advances in Neural Information Processing Systems, vol. 36, 2024. Available: https://proceedings.neurips.cc/paper_files/paper/2023/hash/575c450013d0e99e4b0ecf82bd1afaa4-Abstract-Conference.html
J. Kirchenbauer, J. Geiping, Y. Wen, J. Katz, I. Miers, and T. Goldstein, “A watermark for large language models,” in International Conference on Machine Learning, 2023, pp. 17061–17084. Available: https://proceedings.mlr.press/v202/kirchenbauer23a.html
I. Solaiman et al., “Release strategies and the social impacts of language models,” ArXiv, 2019. doi: https://doi.org/10.48550/arXiv.1908.09203
R. Zellers et al., “Defending against neural fake news,” Advances in Neural Information Processing Systems, vol. 32, 2019. Available: https://proceedings.neurips.cc/paper/2019/hash/3e9f0fc9b2f89e043bc6233994dfcf76-Abstract.html
X. Yu et al., “GPT paternity test: GPT generated text detection with GPT genetic inheritance,” CoRR, 2023. Available: https://arxiv.org/pdf/2305.12519v2
X. Hu, P.-Y. Chen, and T.-Y. Ho, “Radar: Robust AI-text detection via adversarial learning,” Advances in Neural Information Processing Systems, vol. 36, pp. 15077–15095, 2023. Available: https://proceedings.neurips.cc/paper_files/paper/2023/hash/30e15e5941ae0cdab7ef58cc8d59a4ca-Abstract-Conference.html
Y. Tian et al., “Multiscale positive-unlabeled detection of AI-generated texts,” ArXiv, 2023. Available: https://arxiv.org/pdf/2305.18149
V. Verma, E. Fleisig, N. Tomlin, and D. Klein, “Ghostbuster: Detecting text ghostwritten by large language models,” ArXiv, 2023. doi: https://doi.org/10.48550/arXiv.2305.15047
J. Pu, Z. Huang, Y. Xi, G. Chen, W. Chen, and R. Zhang, “Unraveling the mystery of artifacts in machine generated text,” in Proceedings of the Thirteenth Language Resources and Evaluation Conference, pp. 6889–6898, 2022. Available: https://aclanthology.org/2022.lrec-1.744
C. Vasilatos, M. Alam, T. Rahwan, Y. Zaki, and M. Maniatakos, “HowkGPT: Investigating the detection of ChatGPT-generated university student homework through context-aware perplexity analysis,” ArXiv, 2023. doi: https://doi.org/10.48550/arXiv.2305.18226
Y. Wang et al., “M4: Multi-generator, multi-domain, and multi-lingual black-box machine-generated text detection,” ArXiv, 2023. doi: https://doi.org/10.48550/arXiv.2305.14902
E. Mitchell, Y. Lee, A. Khazatsky, C.D. Manning, and C. Finn, “DetectGPT: Zero-shot machine-generated text detection using probability curvature,” in International Conference on Machine Learning, pp. 24950–24962, 2023. Available: https://proceedings.mlr.press/v202/mitchell23a.html
J. Su, T.Y. Zhuo, D. Wang, and P. Nakov, “DetectLLM: Leveraging log rank information for zero-shot detection of machine-generated text,” ArXiv, 2023. doi: https://doi.org/10.48550/arXiv.2306.05540
E. Tulchinskii et al., “Intrinsic dimension estimation for robust detection of AI-generated texts,” Advances in Neural Information Processing Systems, vol. 36, 2024. Available: https://proceedings.neurips.cc/paper_files/paper/2023/hash/7baa48bc166aa2013d78cbdc15010530-Abstract-Conference.html
X. Yang, W. Cheng, Y. Wu, L. Petzold, W.Y. Wang, and H. Chen, “DNA-GPT: Divergent N-Gram Analysis for Training-Free Detection of GPT-Generated Text,” ArXiv, 2023. doi: https://doi.org/10.48550/arXiv.2305.17359
S.S. Ghosal, S. Chakraborty, J. Geiping, F. Huang, D. Manocha, and A.S. Bedi, “Towards possibilities & impossibilities of AI-generated text detection: a survey,” ArXiv, 2023. doi: https://doi.org/10.48550/arXiv.2310.15264
R. Tang, Y.-N. Chuang, and X. Hu, “The science of detecting LLM-generated text,” Communications of the ACM, vol. 67, issue 4, pp. 50–59, 2024. doi: 10.1145/3624725
M. Dhaini, W. Poelman, and E. Erdogan, “Detecting ChatGPT: A survey of the state of detecting ChatGPT-generated text,” ArXiv, 2023. doi: https://doi.org/10.48550/arXiv.2309.07689
B. Guo et al., “How close is ChatGPT to human experts? Comparison corpus, evaluation, and detection,” ArXiv, 2023. doi: https://doi.org/10.48550/arXiv.2301.07597
R. Varshney, N.S. Keskar, and R. Socher, “Limits of detecting text generated by large-scale language models,” in 2020 Information Theory and Applications Workshop (ITA), 2020, pp. 1–5. doi: 10.1109/ITA50056.2020.9245012
H. Helm, C.E. Priebe, and W. Yang, “A Statistical Turing Test for Generative Models,” ArXiv, 2023. doi: https://doi.org/10.48550/arXiv.2309.08913
V.S. Sadasivan, A. Kumar, S. Balasubramanian, W. Wang, and S. Feizi, “Can AI-generated text be reliably detected?,” ArXiv, 2023. doi: https://doi.org/10.48550/arXiv.2303.11156
S. Chakraborty, A.S. Bedi, S. Zhu, B. An, D. Manocha, and F. Huang, “On the possibilities of AI-generated text detection,” ArXiv, 2023. doi: https://doi.org/10.48550/arXiv.2304.04736
L. Li, Z. A. Din, Z. Tan, S. London, T. Chen, and A. Daptardar, “MerRec: A Large-scale Multipurpose Mercari Dataset for Consumer-to-Consumer Recommendation Systems,” ArXiv, 2024. doi: https://doi.org/10.48550/arXiv.2402.14230
R. J. Gallagher et al., “Generalized word shift graphs: a method for visualizing and explaining pairwise comparisons between texts,” EPJ Data Science, vol. 10, no. 1, Jan. 2021. doi: 10.1140/epjds/s13688-021-00260-3
