Generative AI in the Data Scientist’s Universe: A Friendly Companion or a Sneaky Enemy?

2

October

2023

5/5 (1)

In today’s business landscape, data is key. The widespread adoption of Artificial Intelligence (AI) and Machine Learning (ML) has sparked a compelling debate about the future of the data scientist’s role. As we progress into an era where being code literate and possessing technical knowledge are no longer barriers to entry, the accessibility of generative AI tools has transcended the boundaries of academic research and tech giants. It’s now within reach for anyone with an internet connection and an email address.

It made me wonder if this transformation means the death of data scientist-like professions or whether it signifies the birth of new roles. I will delve into this question using my personal experience as a starting point.  

The Learning Curve

About a year ago I started in a newly created role as a junior business controller. I quickly realized the importance of harnessing data to create insightful dashboards that guide data-driven decision-making. Learning to navigate the complex world of data was a challenge in itself (I mean, how do I know that I can trust my data? What data is available and what is relevant?), but the real game-changer came when I discovered the power of generative AI.

It was like having a 24/7 coding mentor at my fingertips

Starting from scratch with SQL was no walk in the park. I encountered countless errors, syntax hiccups, and moments of sheer frustration. But I followed some courses, watched YouTube videos and found out that SQL is amazing (honestly: I would highly recommend learning it)! It was during this process that I stumbled upon an invaluable ally: ChatGPT. Whenever I hit a roadblock, I would turn to ChatGPT and put my code-related questions in. Within seconds, I’d receive clear and concise explanations, troubleshooting tips, and even code snippets to resolve my issues. It was like having a 24/7 coding mentor at my fingertips.

ChatGPT can even help write queries from scratch! Source: https://blog.devart.com/how-to-use-chatgpt-to-write-sql-join-queries.html

The Role of Generative AI in Data-Related Work

Generative AI, like ChatGPT, is not just a tool for beginners like me. I’m convinced that it has the potential to revolutionize the way data scientists and coders work. But should we see it as a friendly companion or a sneaky enemy that is stealing jobs?

AI as a Productivity Booster

First and foremost, generative AI can significantly enhance the productivity of data professionals. It excels at automating repetitive tasks, such as data retrieval and cleaning as well as tackling error messages. This frees up valuable time for more critical analysis. It can also serve as a valuable learning resource, providing instant answers and explanations for coding queries. Due to this I have experienced a steep learning curve in my SQL journey.

Human data scientists and coders bring a unique skill set to the table; they possess the ability to interpret, contextualize, and make nuanced decisions based on data

The Future of Data-Related Jobs

The future of data-related jobs stands at a crossroads, where the demand for AI-ready professionals who can effectively harness tools like ChatGPT is already evident in job descriptions. The fear of AI replacing human expertise is a legitimate concern, but it’s important to remember that AI is a tool, not a replacement. While generative AI offers substantial productivity gains, it’s not immune to errors or hallucinations, reminding us of the irreplaceable value of deep domain expertise. Human data scientists and coders bring a unique skill set to the table; they possess the ability to interpret, contextualize, and make nuanced decisions based on data. AI can undoubtedly assist in the technical aspects of the job, streamlining processes and automating tasks, but it cannot replicate the human touch required for holistic and insightful data analysis.

Well, the way I see it..

In my role, generative AI has truly been a game-changer. It’s become an ally helping me out with coding challenges and making my dashboards way better and more trustworthy. It surprised me by getting results faster than I expected, and I can’t complain about that! And whenever my SQL query was actually running properly, I put it in and asked for suggestions to improve performance. Using ChatGPT for my coding-related questions has boosted my efficiency and effectiveness, and I see it as a sneak peek into the future of data work. It’s all about humans and AI teaming up, playing to each other’s strengths for some seriously awesome outcomes.

So, the next time you encounter a coding conundrum, don’t hesitate to turn to generative AI. It’s not a replacement for your expertise but a trusted partner in your data-driven journey.

Curious to hear your thoughts and experiences! Also let me know whether some of you use other GAI tools for your data- and coding-related issues! 😉

Please rate this

How Greece used AI to detect asymptomatic travelers infected with COVID-19

29

September

2021

No ratings yet.


A few months after the Covid-19 outbreak, operations researcher Kimon Drakopoulos, who works in data science at the University of Southern California, offered to help the Greek government by developing a system that uses machine learning in order to determine which travelers had the most risk of being infected and thus should get tested. Greece was asked by the European Union to allow non-essential travel again, but of course the option of testing all travelers was not available. Consequently, they chose to implement a more efficient way to test incoming travelers than the usual practices of randomized sample testing or testing based on the visitor’s country of origin, by launching this system called ‘Eva’ and deploying it across all Greek borders.

Drakopoulos and his colleagues discovered that machine learning proved to be more effective at identifying asymptomatic cases than the aforementioned methods, by a factor of two to four times during peak tourist season. This was accomplished because Eva used multiple sources of data, besides just travel history, to assess and estimate the infection risk of an individual. These sources include demographic data like the age and sex of the travelers, which was then paired with the obtained data from previously tested passengers, to calculate who had the highest risk out of a group and needed to be tested. This process was also used to provide information to the border policies about real-time estimates of the prevalence of COVID-19.

When the researchers compared the performance of this model against the methods that only use epidemiological metrics, such as random testing, it was clear that it performed better in all aspects. One main reason for this was the limited predictive value that these metrics possessed in relation to asymptomatic cases. Consequently, the paper raises concern on the effectiveness of internationally proposed border policies that employ such population-level metrics.

All in all, Eva is a successful example of how the use of reinforcement learning and artificial intelligence in combination with real-time data can provide very useful assistance both in crisis situations but also in the public health sector.

References

Bastani, H., Drakopoulos, K., Gupta, V. et al. Efficient and targeted COVID-19 border testing via reinforcement learning. Nature (2021). https://doi.org/10.1038/s41586-021-04014-z

Nature (2021) ‘Greece used AI to curb COVID: what other nations can learn’, 22 September. Available at: https://www.nature.com/articles/d41586-021-02554-y  (Accessed: 29 September 2021).

Please rate this

Can machines replace doctors anytime soon?

11

October

2017

No ratings yet. With each day some new developments in data science and machine learning are being published. The researchers constantly invent new ways to improve the performance of their predictive algorithms, thus improving the accuracy of the computers’ predictions on any given subject. One of such subjects, which should be of particular interest to us – humans – is medicine.

With the constant increase in the computation power and exponentially growing amounts of data, the researchers could potentially save a lot of lives with the use of advanced data science solutions. For example, research presented in the journal of the American Academy of Neurology states that the potential for artificial intelligence in precision medicine is significant. IBM Watson, a question answering computer system, provided a report of actionable insights within 10 minutes, in comparison to 160 hours of human analysis normally necessary to reach analogical conclusions (Monegain, 2017). This can often be of crucial difference when dealing with illnesses such as malign cancers, as some of them have a median survival of less than a few months following diagnosis.

Aside from genome analysis, there are countless areas in medicine where machine learning and artificial intelligence solutions can mean a difference between life and death (or very serious health complications). Currently, researchers around the world are trying to incorporate data science applications in detection of diseases such as autism, Parkinson’s, Alzheimer’s to name just a few! All of these sicknesses have one thing in common: early detection always improves the chances for a recovery or prevents further complications. And in this very sense, the AI can help people. Not only it is able to recognize the symptoms of a disease faster that a human can, but also it is able to do so at a larger scale, providing necessary diagnosis to the people in need. For instance, Google is launching an experiment to use machine learning to discover a diabetes-related eye disease in India, where the number of people with diabetes is around 70 million (with approximately 400 million people worldwide) and not a large percentage of them would normally receive a proper diagnosis in time (Simonite, 2017).

Another important factor, also connected to the accessibility, is the fact that a lot of people must postpone visits to the doctors because they cannot afford a private visit, while the wait time for a public one can even be around a year. And with the use of computers the time needed for an accurate diagnosis, as well as the number of specialists to be seen before obtaining it, is drastically reduced (Molteni, 2017).

Of course there are disadvantages of using AI in medicine. Firstly, there is a rising fear among the doctors that they will lose their jobs and be replaced by machines. According to an article in the New England Journal of Medicine, radiology and pathology are primarily susceptible to the power of AI, due to the fact that these jobs are based on pattern matching and machines can perform such tasks with surprising accuracy and speed (Asay, 2017). However, AI should mostly complement the work of doctors, enable them to perform their job more efficiently, thus help more people. Secondly, there are many issues regarding the data. Some people are not always aware that by signing a particular form they enable a company (or companies) to use their private data for research purpose (Molteni, 2017). What is more, for the algorithms to have a high accuracy, they need to have a lot of data at their disposal, which is not always feasible. For example, in the process of detecting the case of autism among kids, a lot of data needs to be gathered from MRI scans. This is not only time-consuming, but also expensive procedure. And it would not be realistic to have every child scanned in order to have a sufficient dataset (Vlasits, 2017).

Summing up, the rapid development of artificial intelligence and big data promises a future in which computers will be able to assist the doctors in providing quick and accurate diagnosis, thus saving human lives. However, one cannot forget that the algorithms should assist the doctors, instead of taking over their jobs. There will always be a fear that a machine provides an inaccurate prognosis, which can either scare the patient or make him calm, when he or she can be seriously sick. That is why human input is always valuable, to evaluate the machine’s diagnosis and verify its correctness. In other words, we should be very optimistic about the possibilities offered by artificial intelligence, but at the same time should not expect a bunch of robots treating patients in the hospital anytime soon.

References:

Asay M. (2017, January). Why AI is about to make some of the highest-paid doctors obsolete. Tech Republic, retrieved from: http://www.techrepublic.com/article/why-ai-is-about-to-make-some-of-the-highest-paid-doctors-obsolete/

Molteni, M. (2017, August). Want a diagnosis tomorrow, not next year? Turn to AI. Wired, retrieved from: https://www-wired-com.eur.idm.oclc.org/story/ai-that-will-crowdsource-your-next-diagnosis/

Molteni, M. (2017, September). 23andme is digging through your data for a Parkinson’s cure. Wired, retrieved from: https://www-wired-com.eur.idm.oclc.org/story/23andme-is-digging-through-your-data-for-a-parkinsons-cure/

Monegain, B. (2017, July). AI can speed up precision medicine, New York Genome Center-IBM Watson study shows. Healthcareitnews, retrieved from: http://www.healthcareitnews.com/news/ai-can-speed-precision-medicine-new-york-genome-center-ibm-watson-study-shows

Simonite, T. (2017, June). Google’s AI Eye Doctor Gets Ready to Go to Work in India. Wired, retrieved from: https://www.wired.com/2017/06/googles-ai-eye-doctor-gets-ready-go-work-india/

Vlasits, A. (2017, June). AI could target autism before it even emerges – but it’s no cure-all. Wired, retrieved from: https://www-wired-com.eur.idm.oclc.org/story/ai-could-target-autism-before-it-even-emerges-but-its-no-cure-all/

Please rate this