research-article
Authors: Chuhan Shi, Weiwei Cui, Chengzhong Liu, Chengbo Zheng, + 3, Haidong Zhang, Qiong Luo, Xiaojuan Ma (Less)
IEEE Transactions on Visualization and Computer Graphics, Volume 30, Issue 1
Pages 814 - 824
Published: 23 October 2023 Publication History
Metrics
Total Citations2Total Downloads0Last 12 Months0
Last 6 weeks0
New Citation Alert added!
This alert has been successfully added and will be sent to:
You will be notified whenever a record that you have chosen has been cited.
To manage your alert preferences, click on the button below.
Manage my Alerts
New Citation Alert!
Please log in to your account
- View Options
- References
- Media
- Tables
- Share
Abstract
Choice of color is critical to creating effective charts with an engaging, enjoyable, and informative reading experience. However, designing a good color palette for a chart is a challenging task for novice users who lack related design expertise. For example, they often find it difficult to articulate their abstract intentions and translate these intentions into effective editing actions to achieve a desired outcome. In this work, we present NL2Color, a tool that allows novice users to refine chart color palettes using natural language expressions of their desired outcomes. We first collected and categorized a dataset of 131 triplets, each consisting of an original color palette of a chart, an editing intent, and a new color palette designed by human experts according to the intent. Our tool employs a large language model (LLM) to substitute the colors in original palettes and produce new color palettes by selecting some of the triplets as few-shot prompts. To evaluate our tool, we conducted a comprehensive two-stage evaluation, including a crowd-sourcing study (<inline-formula><tex-math notation="LaTeX">$\mathrm{N}=71$</tex-math><alternatives><inline-graphic xlink:href="tvcg-shi-3326522-eqinline-1-small.tif"/></alternatives></inline-formula>) and a within-subjects user study (<inline-formula><tex-math notation="LaTeX">$\mathrm{N}=12$</tex-math><alternatives><inline-graphic xlink:href="tvcg-shi-3326522-eqinline-2-small.tif"/></alternatives></inline-formula>). The results indicate that the quality of the color palettes revised by NL2Color has no significantly large difference from those designed by human experts. The participants who used NL2Color obtained revised color palettes to their satisfaction in a shorter period and with less effort.
References
[1]
Adobe Color. [Online]. Available: https://color.adobe.com/, Accessed in 2023.
[2]
Adobe Illustrator. [Online]. Available: https://www.adobe.com/products/illustrator/ Accessed in 2023.
[3]
Adobe Photoshop. [Online]. Available: https://www.adobe.com/products/photoshop/, Accessed in 2023.
[4]
Color Hunt. [Online]. Available: https://colorhunt.co/, Accessed in 2023.
[5]
Coolors. [Online]. Available: https://coolors.co/, Accessed in 2023.
[6]
Inkscape. [Online]. Available: https://inkscape.org/, Accessed in 2023.
[7]
Material UI. [Online]. Available: https://materialui.co/, Accessed in 2023.
[8]
Palettable. [Online]. Available: https://www.palettable.io/, Accessed in 2023.
[9]
Vega-Lite. [Online]. Available: https://vega.github.io/vega-lite/examples/, Accessed in 2023.
[10]
H. Bahng, S. Yoo, W. Cho, D. K. Park, Z. Wu, X. Ma, and J. Choo. Coloring with words: Guiding image colorization through text-based palette generation. In Proceedings of the european conference on computer vision (eccv), pp. 431–447, 2018.
[11]
L. Bartram, A. Patra, and M. Stone. Affective color in visualization. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, pp. 1364–1374, 2017.
Digital Library
[12]
M. A. Borkin, A. A. Vo, Z. Bylinskii, P. Isola, S. Sunkavalli, A. Oliva, and H. Pfister. What makes a visualization memorable? IEEE transactions on visualization and computer graphics, 19 (12): pp. 2306–2315, 2013.
Digital Library
[13]
C. Brewer. Color use guidelines for mapping and visualization. visualization in modern cartography, 1994.
[14]
T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell et al., Language models are few-shot learners. in Advances in neural information processing systems, 33: pp. 1877–1901, 2020.
[15]
W. Cui, X. Zhang, Y. Wang, H. Huang, B. Chen, L. Fang, H. Zhang, J.-G. Lou, and D. Zhang. Text-to-viz: Automatic generation of infographics from proportion-related natural language statements. IEEE transactions on visualization and computer graphics, 26 (1): pp. 906–916, 2019.
[16]
Y. Feng, X. Wang, B. Pan, K. K. Wong, Y. Ren, S. Liu, Z. Yan, Y. Ma, H. Qu, and W. Chen. Xnli: Explaining and diagnosing nli-based visual data analysis. IEEE Transactions on Visualization and Computer Graphics, 2023.
[17]
M. Harrower and C. A. Brewer. Colorbrewer. org: an online tool for selecting colour schemes for maps. The Cartographic Journal, 40 (1): pp. 27–37, 2003.
[18]
S. G. Hart and L. E. Staveland. “Development of nasa-tlx (task load index): Results of empirical and theoretical research”. In Advances in psychology, vol. 52, pp. 139–183. Elsevier, 1988.
[19]
M. Hearst, M. Tory, and V. Setlur. Toward interface defaults for vague modifiers in natural language interfaces for visual analysis. In 2019 IEEE Visualization Conference (VIS), pp. 21–25. IEEE, 2019.
[20]
K. Hu, M. A. Bakker, S. Li, T. Kraska, and C. Hidalgo. Vizml: A machine learning approach to visualization recommendation. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, pp. 1–12, 2019.
Digital Library
[21]
H. Kaur, A. Williams, and W. Lasecki. Building shared mental models between humans and ai for effective collaboration. in CHI'19, May 2019, Glasgow, Scotland, 2019.
[22]
T. Kenter and M. De Rijke. Short text similarity with word embeddings. In Proceedings of the 24th ACM international on conference on information and knowledge management, pp. 1411–1420, 2015.
Digital Library
[23]
T. S. Kim, D. Choi, Y. Choi, and J. Kim. Stylette: Styling the web with natural language. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems, pp. 1–17, 2022.
Digital Library
[24]
M. Krause, T. Garncarz, J. Song, E. M. Gerber, B. P. Bailey, and S. P. Dow. Critique style guide: Improving crowdsourced design feedback with a natural language model. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, pp. 4627–4639, 2017.
[25]
Y. Li, S. Sarcar, Y. Zheng, and X. Ren. Exploring text revision with backspace and caret in virtual reality. In Proceedings of the 2021 CHI conference on human factors in computing systems, pp. 1–12, 2021.
Digital Library
[26]
S. Lin, J. Fortuna, C. Kulkarni, M. Stone, and J. Heer. “Selecting semantically-resonant colors for data visualization”. In Computer Graphics Forum, vol. 32, pp. 401–410. Wiley Online Library, 2013.
[27]
C. Liu, Y. Han, R. Jiang, and X. Yuan. Advisor: Automatic visualization answer for natural-language question on tabular data. In 2021 IEEE 14th Pacific Visualization Symposium (PacificVis), pp. 11–20. IEEE, 2021.
[28]
H. Lü and Y. Li. Gesture coder: a tool for programming multi-touch gestures by demonstration. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 2875–2884, 2012.
Digital Library
[29]
Y. Luo, N. Tang, G. Li, J. Tang, C. Chai, and X. Qin. Natural language to visualization by neural machine translation. IEEE Transactions on Visualization and Computer Graphics, 28 (1): pp. 217–226, 2021.
Digital Library
[30]
B. J. Meier, A. M. Spalter, and D. B. Karelitz. Interactive color palette tools. IEEE Computer Graphics and Applications, 24 (3): pp. 64–72, 2004.
Digital Library
[31]
S. Mishra, D. Khashabi, C. Baral, Y. Choi, and H. Hajishirzi. Reframing instructional prompts to gptk's language. arXiv preprint arXiv:, 2021.
[32]
I. Pavan Kumar, V. Hara Gopal, S. Ramasubbareddy, S. Nalluri, and K. Govinda. Dominant color palette extraction by k-means clustering algorithm and reconstruction of image. In Data Engineering and Communication Technology: Proceedings of 3rd ICDECT-2K19, pp. 921–929. Springer, 2020.
[33]
Y.-F. Peng and T.-R. Chou. Automatic color palette design using color image and sentiment analysis. In 2019 IEEE 4th International Conference on Cloud Computing and Big Data Analysis (ICCCBDA), pp. 389–392. IEEE, 2019.
[34]
Q. Qiu, M. Otani, and Y. Iwazaki. An intelligent color recommendation tool for landing page design. In 27th International Conference on Intelligent User Interfaces, pp. 26–29, 2022.
Digital Library
[35]
Z. Qu and J. Hullman. Keeping multiple views consistent: Constraints, validations, and exceptions in visualization authoring. IEEE transactions on visualization and computer graphics, 24 (1): pp. 468–477, 2017.
[36]
M. M. Rashid, H. K. Jahan, A. Huzzat, R. A. Rahul, T. B. Zakir, F. Meem, M. S. H. Mukta, and S. Shatabda. Text2chart: A multi-staged chart generator from natural language text. In Advances in Knowledge Discovery and Data Mining: 26th Pacific-Asia Conference, PAKDD 2022, Chengdu, China, May 16–19, 2022, Proceedings, Part II, pp. 3–16. Springer, 2022.
[37]
L. Reynolds and K. McDonell. Prompt programming for large language models: Beyond the few-shot paradigm. In Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems, pp. 1–7, 2021.
Digital Library
[38]
K. B. Schloss, C. C. Gramazio, A. T. Silverman, M. L. Parker, and A. S. Wang. Mapping color to meaning in colormap data visualizations. IEEE transactions on visualization and computer graphics, 25 (1): pp. 810–819, 2018.
[39]
V. Setlur and A. Kumar. Sentifiers: Interpreting vague intent modifiers in visual analysis using word co-occurrence and sentiment analysis. In 2020 IEEE Visualization Conference (VIS), pp. 216–220, 2020.
[40]
V. Setlur and M. C. Stone. A linguistic approach to categorical color assignment for data visualization. IEEE transactions on visualization and computer graphics, 22 (1): pp. 698–707, 2015.
[41]
M. Shugrina, W. Zhang, F. Chevalier, S. Fidler, and K. Singh. Color builder: A direct manipulation interface for versatile color theme authoring. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, pp. 1–12, 2019.
Digital Library
[42]
S. Smart, K. Wu, and D. A. Szafir. Color crafting: Automating the construction of designer quality color ramps. IEEE transactions on visualization and computer graphics, 26 (1): pp. 1215–1225, 2019.
[43]
D. So, W. Mańke, H. Liu, Z. Dai, N. Shazeer, and Q. V. Le. Searching for efficient transformers for language modeling. in Advances in Neural Information Processing Systems, 34: pp. 6010–6022, 2021.
[44]
M. K. Stern and J. H. Johnson. Just noticeable difference. The Corsini Encyclopedia of Psychology, pp. 1–2, 2010.
[45]
C. Tseng, G. J. Quadri, Z. Wang, and D. A. Szafir. Measuring categorical perception in color-coded scatterplots. In proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, pp. 1–14, 2023.
Digital Library
[46]
M. Tuli, A. Li, P. Vaezipoor, T. Klassen, S. Sanner, and S. McIlraith. Learning to follow instructions in text-based games. in Advances in Neural Information Processing Systems, 35: pp. 19441–19455, 2022.
[47]
H. Voigt, Ö. Alaçam, M. Meuschke, K. Lawonn, and S. Zarrieß. The why and the how: A survey on natural language interaction in visualization. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 348–374, 2022.
[48]
T. vor der Brück and M. Pouly. Text similarity estimation based on word embeddings and matrix norms for targeted marketing. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pp. 1827–1836, 2019.
[49]
Y. Wang, Z. Hou, L. Shen, T. Wu, J. Wang, H. Huang, H. Zhang, and D. Zhang. Towards natural language-based visualization authoring. IEEE Transactions on Visualization and Computer Graphics, 29 (1): pp. 1222–1232, 2022.
[50]
M. Wijffelaars, R. Vliegen, J. J. Van Wijk, and E.-J. Van Der Linden. “Generating color palettes using intuitive parameters”. In Computer Graphics Forum, vol. 27, pp. 743–750. Wiley Online Library, 2008.
[51]
T. Wu, M. Terry, and C. J. Cai. Ai chains: Transparent and controllable human-ai interaction by chaining large language model prompts. In Proceedings of the 2022 CHI conference on human factors in computing systems, pp. 1–22, 2022.
Digital Library
[52]
L. Yan, E. L. Glassman, and T. Zhang. Visualizing examples of deep neural networks at scale. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, pp. 1–14, 2021.
Digital Library
[53]
Z. Yang, Z. Gan, J. Wang, X. Hu, Y. Lu, Z. Liu, and L. Wang. An empirical study of gpt-3 for few-shot knowledge-based vqa. In Proceedings of the AAAI Conference on Artificial Intelligence, vol. 36, pp. 3081–3089, 2022.
[54]
B. Yu and C. T. Silva. Flowsense: A natural language interface for visual data exploration within a dataflow system. IEEE transactions on visualization and computer graphics, 26 (1): pp. 1–11, 2019.
[55]
L.-P. Yuan, Z. Zhou, J. Zhao, Y. Guo, F. Du, and H. Qu. Infocolorizer: Interactive recommendation of color palettes for infographics. IEEE Transactions on Visualization and Computer Graphics, 28 (12): pp. 4252–4266, 2021.
[56]
J. Zamfirescu-Pereira, R. Y. Wong, B. Hartmann, and Q. Yang. Why johnny can't prompt: how non-ai experts try (and fail) to design llm prompts. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, pp. 1–21, 2023.
Digital Library
[57]
A. Zeileis, J. C. Fisher, K. Hornik, R. Ihaka, C. D. McWhite, P. Murrell, R. Stauffer, and C. O. Wilke. colorspace: A toolbox for manipulating and assessing colors and palettes. arXiv preprint arXiv:, 2019.
Cited By
View all
- Hou YYang MCui HWang LXu JZeng W(2024)C2Ideas: Supporting Creative Interior Color Design Ideation with a Large Language ModelProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642224(1-18)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642224
- McNutt AStone MHeer J(2024)Mixing Linters with GUIs: A Color Palette Design ProbeIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2024.345631731:1(327-337)Online publication date: 11-Sep-2024
https://dl.acm.org/doi/10.1109/TVCG.2024.3456317
Index Terms
NL2Color: Refining Color Palettes for Charts with Natural Language
Applied computing
Index terms have been assigned to the content through auto-classification.
Recommendations
- Aesthetic Rating and Color Suggestion for Color Palettes
A model to rate color combinations that considers human aesthetic preferences is proposed. The proposed method does not assume that a color palette has a specific number of colors, i.e., input is not restricted to a two-, three-, or five-color palettes. ...
Read More
- Color Palettes Overview After Thresholding Process with Default Methods of ImageJ or FIJI∗
ICONETSI '22: Proceedings of the 2022 International Conference on Engineering and Information Technology for Sustainable Industry
Coloring techniques in life sciences continue to progress, which have now used deep learning in the staining stage to create virtual staining. Usually, manual staining produces an image with a more vigorous color intensity than virtual staining. This ...
Read More
- Dynamic closest color warping to sort and compare palettes
A color palette is one of the simplest and most intuitive descriptors that can be extracted from images or videos. This paper proposes a method to assess the similarity between color palettes by sorting colors. While previous palette similarity measures ...
Read More
Comments
Information & Contributors
Information
Published In
IEEE Transactions on Visualization and Computer Graphics Volume 30, Issue 1
Jan. 2024
1456 pages
Issue’s Table of Contents
1077-2626 © 2023 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See https://www.ieee.org/publications/rights/index.html for more information.
Publisher
IEEE Educational Activities Department
United States
Publication History
Published: 23 October 2023
Qualifiers
- Research-article
Contributors
Other Metrics
View Article Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- View Citations
2
Total Citations
Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 01 Dec 2024
Other Metrics
View Author Metrics
Citations
Cited By
View all
- Hou YYang MCui HWang LXu JZeng W(2024)C2Ideas: Supporting Creative Interior Color Design Ideation with a Large Language ModelProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642224(1-18)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642224
- McNutt AStone MHeer J(2024)Mixing Linters with GUIs: A Color Palette Design ProbeIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2024.345631731:1(327-337)Online publication date: 11-Sep-2024
https://dl.acm.org/doi/10.1109/TVCG.2024.3456317
View Options
View options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in
Full Access
Get this Publication
Media
Figures
Other
Tables
