In an exciting announcement to close out the year, Scale has been featured in two of Gartner, Inc.’s 2021 Hype Cycles. We are honored to have been featured in Hype Cycles for both Artificial Intelligence and Data Science and Machine Learning. These reports detail emerging technologies in AI, data science and machine learning, and name Scale as a representative vendor for Data Labeling and Annotation Services. The reports describe an urgency for data labeling and annotation services (DLA) to become widely adopted. According to Gartner analyst Anthony Mullen’s analysis, the demand for labeled and annotated data has risen in order to remove bottlenecks in industry-specific use cases. And, due to the fact that most companies do not have in-house skills and processes for DLA, companies like Scale are the best option to provide DLA to yield optimal AI results. Why good data matters Better data leads to more performant models, and performant models lead to faster deployment. With better data, Scale helps customers deliver more value. An increased diversity of use cases in DLA will drive both AI and industry-specific innovation, according to Gartner analyst Anthony Mullen. Vendors in the market have specific offerings across industries including commerce, robotics, autonomous vehicles, retail, GiS/maps, AR/VR, agriculture, finance, transportation and more. Scale is proud to accelerate innovation for customers including PayPal, iRobot, Square, Brex, Pinterest, Etsy, SAP, Airbnb, Instacart, NVIDIA, OpenAI, Samsung, Toyota, GM, the U.S. Air Force, and more. How does Scale solve the bad data problem? Scale is your AI readiness partner. We got our start in data labeling and annotation for autonomous vehicles. Now, Scale provides a data-centric, end-to-end solution to manage the entire ML lifecycle. That means that Scale can meet you wherever you are in your data process. ML-powered, human-in-the loop data Scale’s strength is the combined power of our ML platform and expert human insights to help our customers develop their data strategy with high quality data from the ground up. According to Mullen, there are AI solutions which would be impossible without human-in-the-loop labeling, including in cases of search engine tuning and image recognition. Furthermore, DLA allows data scientists to allocate and prioritize time for higher-order tasks, instead of taking time and effort to annotate data themselves. With Scale Rapid, any machine learning developer can achieve production-quality labels with no data minimums. Receive quality labels in just a matter of hours, with detailed insights and identified edge cases. Scale Rapid can even handle subjective tasks with the utmost precision and quality. But what comes after labeled data? Gartner Inc., has made the case for why data matters to AI development. After data labeling, what’s next? Data management. Whether you have prepared the data, or we label and annotate the data for you, we’re your premiere data management partner. Scale Nucleus is the command and control for your ML dataset. Visualize, curate and debug your dataset to improve your ML model performance with Nucleus. Scale’s advantage: AI development, from data labeling, through the entire ML lifecycle. Scale’s mission is to accelerate the development of AI. We do this by providing a data-centric, end-to-end solution to annotate, manage, automate, evaluate, collect and generate your data. Our products and services meet you wherever you are in the ML lifecycle, whether you need data labeled and annotated, or need an ML engineer to help explore and refine how AI can help your business.

Gartner Hype Cycles for AI and Data Science and Machine Learning

Scale highlighted in Gartner Inc.’s 2021 AI Hype Cycles

Scale Highlighted in Gartner Inc.’s 2021 AI Hype Cycles

Innovations in the field of Machine Learning (ML) continue to be driven by a dynamic research community. The ML team at Scale is proud to contribute to the field through its own research, dataset partnerships with leading Universities such as Oxford (<a href="https://scale.com/open-datasets/oxford" rel="noopener noreferrer" target="_blank">DEBAGREEMENT: Reddit 50K Dataset</a>) and the University of Waterloo and the University of Toronto (<a href="https://scale.com/open-datasets/oxford" rel="noopener noreferrer" target="_blank">CADC Dataset</a>), and collaborations with Stanford University, the University of California, Berkeley, and the University of Washington. As <a href="https://neurips.cc/" rel="noopener noreferrer" target="_blank">NeurIPS 2021</a>, gets underway, we are pleased to share details on three papers the ML team have published as part of the conference: <ol><li> </li><li>DEBAGREEMENT: A Comment-Reply Dataset for (Dis)agreement Detection in Online Debates</li><li>Authors: John Pougué-Biyong, Valentina Semenova, Alexandre Matton, Rachel Han, Aerin Kim, Renaud Lambiotte, Doyne Farmer</li><li> </li><li>Natural Adversarial Objects</li><li>Authors: Felix Lau, Sasha Harrison, Nishant Subramani, Aerin Kim, Elliot Branson, Rosanne Liu</li><li> </li><li> </li><li>Are We Learning Yet? A Meta-Review of Evaluation Failures Across Machine Learning</li><li>Authors: Thomas Liao, Rohan Taori, Inioluwa Deborah Raji, Ludwig Schmidt</li><li> </li></ol><h2><a href="https://openreview.net/forum?id=udVUN__gFO" rel="noopener noreferrer" target="_blank">DEBAGREEMENT: A Comment-Reply Dataset for (Dis)agreement Detection in Online Debates</a></h2> <h3>Paper Summary:</h3> DEBAGREEMENT is a dataset including 42,894 comment-reply pairings from the popular debate website Reddit, each of which has been labeled with agree, neutral, or disagree. We gathered interactions from five different forums: r/BlackLivesMatter, r/Brexit, r/Climate, r/Democrats, and r/Republican. Comment pairings for each forum were chosen in such a way that they generate a user interaction graph when taken as a whole. DEBAGREEMENT offers a challenge for Natural Language Processing (NLP) systems because it includes slang, irony, and topic-specific humor, all of which are common in online discussions. We compared the performance of state-of-the-art language models on a (dis)agreement detection task, and looked at the usage of contextual information that is accessible to the models during training (graph, authorship, and temporal information). DEBAGREEMENT provides novel opportunities for combining graph-based and text-based machine learning techniques to detect agreements as well as disagreements online, in light of recent research showing that context, such as social context or knowledge graph information, enables language models to perform better on downstream NLP tasks. <h3>Key Takeaways:</h3> <ol><li> </li><li>Language models trained on existing datasets underperform when run on real world data. This discrepancy is exposed when evaluating the models using the new DEBAGREEMENT dataset.</li><li> </li><li> </li><li>DEBAGREEMENT presents new opportunities for modeling diverse online interactions with text and context (authorship, graph, temporal information). Its graph structure allows for the combination of text-based machine learning (ML) with graph representation learning (GRL) approaches.</li><li> </li><li> </li><li>By modeling online discussion forums as graphs of user interactions, researchers can: 1) transform the agreement/disagreement detection problem into a sign link prediction task, and 2) use existing signed graph embedding techniques evaluated on publicly accessible signed graphs such as Epinions and Slashdot.</li><li>The sentiment and polarization of topics is not static in time and for the first time, this dataset exposes that and shows how realistic topic sentiment evolves over time.</li><li> </li></ol><h2><a href="https://arxiv.org/abs/2111.04204" rel="noopener noreferrer" target="_blank">Natural Adversarial Objects</a></h2> <h3>Paper Summary:</h3> Although cutting-edge object identification approaches have shown impressive performance, models are often vulnerable to adversarial assaults and out-of-distribution data. Natural Adversarial Objects (NAO) is a novel dataset for assessing the robustness of object identification techniques. NAO comprises 7,934 images and 9,943 objects that are unaltered and depict real-world circumstances, yet cause cutting-edge detection methods to misclassify with high confidence. When compared to the conventional MS-COCO validation set, the mean average precision (mAP) of EfficientDet-D7 lowers by 74.5 percent when tested on NAO. Furthermore, when evaluating a range of object detection architectures, we discovered that improved performance on the MS-COCO validation set does not always transfer to better performance on the NAO, implying that robustness cannot be gained merely by training a more accurate model. We look at why NAO cases are tough to identify and classify. Experiments using image patch shuffles demonstrate that models are too sensitive to local texture. Furthermore, when predicting class labels with integrated gradients and background replacement, we discovered that the detection model is dependent on pixel information inside the bounding box and insensitive to background context. <h3>Key Takeaways:</h3> <ol><li> </li><li>The NAO dataset presents a difficult robustness assessment challenge for detection models trained on MS-COCO. As part of our evaluation, we tested seven state-of-the-art detection models from a variety of families, and found that they consistently failed to perform accurately on NAO when compared to the MS-COCO validation set, which included both in-distribution and out-of-distribution objects, among other things.</li><li> </li><li> </li><li>We also described the process of building such a dataset yourself, which may be used to create similar datasets for any target dataset or pre-trained model.</li><li> </li><li> </li><li>The popular MS-COCO dataset has "blind-spots" that make it challenging to appropriately categorize these naturally adversarial objects. </li><li> </li><li> </li><li>Additionally, we show that detection models are overly sensitive to local texture but insensitive to background change, which makes them vulnerable to natural adversarial objects.</li><li> </li></ol><h2><a href="https://openreview.net/pdf?id=mPducS1MsEK" rel="noopener noreferrer" target="_blank">Are We Learning Yet? A Meta-Review of Evaluation Failures Across Machine Learning</a></h2> <h3>Paper Summary:</h3> Evaluation issues often undermine the validity of results in machine learning research. In collaboration with researchers from Stanford University, the University of California, Berkeley, and the University of Washington, we conducted a meta-review of 100+ survey papers to identify common benchmark evaluation problems across subfields. In some cases, several years’ worth of progress in certain fields may be misstated. Our meta-review surveys evaluation papers reporting on a broad range of subfields, ranging from computer vision to deep reinforcement learning, to recommender systems and natural language processing, and more. We found a consistent set of failure modes, which we organized into a systematic taxonomy. This taxonomy of failure modes helps researchers understand where different types of failures are common and provides a framework for future research. Our taxonomy distinguishes between “internal” and “external” validity failures. Internal validity failures occur within the context of a single benchmark and raise questions about the consistency or reliability of measurements on that specific benchmark. For example, running baseline methods with less hyperparameter tuning than a newly proposed method can underestimate the performance of existing methods and hence lead to overly optimistic claims of performance improvements. External validity failures, in contrast, describe failures that occur when dealing with multiple benchmarks. External validity failures occur when results from one benchmark do not generalize to other settings. One example is dataset misalignment: if data collected for a benchmark is not representative of the setting where models will be used, then results from that benchmark may not reliably predict performance in the real world. <h3>Key Takeaways:</h3> <ol><li> </li><li>Evaluation failures pose a risk to the validity of research findings in machine learning. Our meta-review of 100+ survey papers observed common failure modes across a wide range of subfields. </li><li> </li><li> </li><li>While some internal validity failures can be identified in the context of a single benchmark, other external validity failures only appear when looking at multiple datasets.</li><li> </li></ol><h2>Conclusion</h2> As we look ahead to 2022, we look forward to supporting more university research teams and driving more research initiatives within Scale. To learn more about our work and the work of many of our partners, our team hosts regular TechTalks on <a href="https://exchange.scale.com/public/events" rel="noopener noreferrer" target="_blank">Scale Exchange</a>, our community of ML practitioners. Join the community to learn more and register for upcoming talks. If you’re interested in joining our growing ML team, take a look at our <a href="https://scale.com/careers" rel="noopener noreferrer" target="_blank">careers page</a> for open positions.

An Overview of ML Research Papers Accepted At NeurIPS 2021

Research Papers Accepted At NeurIPS 2021

Machine Learning Papers at NeurIPS 2021

Artificial Intelligence (AI) is now a part of our everyday lives and has the potential to massively impact the future of our society. Leaders in the private sector are making a real-world impact with AI today - from autonomous vehicles, medical diagnostics, vaccine distribution, and eCommerce. We also recognize the importance of the federal government, academia, and the private sector to work collaboratively to drive innovation and ensure continued U.S. leadership in technology. In early October, we hosted our second virtual conference: TransformX. Over the two days, we brought together more than 22,000 researchers, practitioners, and executives to hear from leaders across industries, academia, and government. We’re excited to share some key insights and takeaways that the foremost government leaders have on the future of AI. <h1>Fireside Chats With Key Leaders</h1>Mark “Drifter” Valentine, Head of Federal at Scale AI hosted fireside discussions with key government and private sector leaders including Lieutenant General Kirk S. Pierce, Commander, Continental U.S. North American Aerospace Defense Command Region-1st Air Force, Rear Admiral Lorin Selby, Chief of Naval Research, and Steven Escaravage, Senior Vice President at Booz Allen Hamilton. <h2>Lieutenant General Kirk S. Pierce</h2> LTG Pierce spoke with Mark on operationalizing AI for National Security. For Pierce, homeland defense is a chief interest. To work effectively and with prudence, “We have this grand, sweeping concept of joint all domain command and control.” To achieve this, Pierce says interoperability is key. The key to interoperability? Data. “We cannot continue to have data stovepipes,” he said. “Depending on wherever that data comes from—whether it's a sensor, above water, below water, in space, whatever it is, we want to have the data to be able to make decisions.” He also said that he sees AI and ML operability as “an opportunity to reset, fundamentally, how we look at information, how we aggregate information and be able to evaluate and assess the validity of the information.” “The companies that are working on that are gonna change how we fundamentally, as senior leaders, make decisions.” <iframe class="ql-video" frameborder="0" allowfullscreen="true" src="https://scaleai.wistia.com/embed/medias/vgxbm41ptn"></iframe> <h2>Rear Admiral Lorin Selby</h2> RADM Selby discussed the importance of fostering an AI innovation ecosystem. As a leader who is positioned at the cross-section of government, academia and industry, Selby is optimistic about the utility of AI to help humans reach their potential, and the Navy achieve its goals. He foresees a future where AI can “unburden the mind” from tedious bureaucratic work, “So we can actually focus on the things the human mind is really good at focusing on— which is kind of more strategic thinking, more abstract thinking.” “This is a pivotal moment in history,” he said of AI and ML advancements. “I think we’ll look back on this time as a key transition phase in history.” Selby said that building confidence in AI systems is essential to innovation, as is remaining anchored in the strength of human decision making. “We have to develop trust,” he said, while emphasizing the need to keep humans in the loop of critical decisions. <iframe class="ql-video" frameborder="0" allowfullscreen="true" src="https://scaleai.wistia.com/embed/medias/h3hikuszel"></iframe> <h2>Steve Escaravage</h2> Steve Escaravage shared his insights on developing realistic approaches to deploying machine learning (ML) in federal environments. Escaravage said that for advancements in ML, the private sector is leading, and the government has to follow. “The innovation is coming from outside—it’s coming from the private sector, from commercial use cases” he said. “For the public sector—for the government to be successful, we simply cannot ignore that investment and we need to channel that into the public sector.” He said that although there has been real progress and momentum since 2017, there are pieces of the puzzle that are missing. Friction points include: access to real world data assets and data sets to train AI, approved computing environments that are configurable and scalable and systems with authorization to operate (ATO) accreditation. “We all have a responsibility as citizens of our governments to try to get them to better outcomes,” he remarked.<iframe class="ql-video" frameborder="0" allowfullscreen="true" src="https://scaleai.wistia.com/embed/medias/0b1pm4gn25"></iframe> <h2>Expert Panel:</h2> In addition to the fireside chats, Mark also led an expert panel discussion on The State of AI for National Security Mission Applications featuring Michael “Rabbi” Harasimowicz and Rachael Martin. Martin said that she hopes that in the future, AI will be a seamless part of her agency’s workflow, and as commonplace, critical and functional as something like email. “The new warfighting demand is time,” she said. “When you have machine speed, you need to have trust and confidence that what the machine is telling you is correct.” As the NGA relies on overhead imagery and data to compel decision-making, she said it’s important to have a systematic data workflow from input to warfighter. “We do think that AI and automation is really the best way of accomplishing that,” Martin said. She added that she sees the time investment and the leadership investment in bringing AI to the mission space as heartening. “You have to change culture by getting people involved in the change,” she reflected. Harasimowicz said that if our government doesn’t update its systems, the systems and the country are at risk. “Our national security challenges are really getting to the point where they're beyond human scale,” he said. “And I say that because the amount of data that is flowing in and around all of our domains is just insurmountable for just a commander.” The level of support required to manage the data at a tactical and strategic level drives us to have a level of assistance, he said. “So artificial intelligence can provide that backdrop.” “Our adversaries are challenging themselves in aggressive ways,” Harasimowicz said. “And we have to be as aggressive, if not more so aggressive…We have to go faster. We have to find ways to solve solutions in weeks, as opposed to years.”<iframe class="ql-video" frameborder="0" allowfullscreen="true" src="https://scaleai.wistia.com/embed/medias/2yn8oyur22"></iframe>

Highlights from TransformX 2021: U.S. Government Related Sessions

Highlights from TransformX 2021: U.S. Government Sessions

<h1>Natural Language Processing and Online Discourse:</h1>Online debates shape our society, and have a profound impact on how people perceive the world. Over 4.2 billion people actively use social media, and many of them participate in online discussions. The breadth and depth of online discourse provide insights and data for researchers and curious people alike.One way to understand data from online discussions is through the field of Natural Language Processing (NLP). NLP offers techniques to understand textual interactions, and to gain a better understanding of polarization online.However, current NLP models often struggle with nuanced language in online exchanges, such as slang, sarcasm and topic-specific jokes, making it difficult to understand diverse online interactions. A task called stance detection is useful in this arena to address this challenge.Stance detection is a text understanding task that consists of classifying the position (or stance) of the author of a piece of text, towards an initial message or statement. In an online setting, these statements might include comments or messages in a thread. The stance is classified into one of three classes: in favor, against, or neutral.<h1>Methodology:</h1>Recent work has shown that stance detection would benefit from context-sensitive approaches. We were not able to find realistic, large, open datasets with contextual information which do not have platform-specific features.Thus, we created <a href="https://scale.com/open-datasets/oxford" rel="noopener noreferrer" target="_blank">DEBAGREEMENT</a> with conversations from Reddit to better understand the evolution of online debates, and to provide <a href="https://scale.com/guides/data-labeling-annotation-guide" rel="noopener noreferrer" target="_blank">annotated data</a> for agreement and disagreement detection.The comment or reply function on social media sites does not provide sufficient information to classify an interaction. So, we looked at actual online conversations. Unlike existing datasets for stance detection, DEBAGREEMENT provides realistic online discussions, with diverse writing styles, genres and topics of discussion.The set is comprised of 42,894 comment-reply pairs from Reddit, annotated with agree, neutral, or disagree labels.The dataset collects data from five forums on Reddit:<ol><li>r/BlackLivesMatter</li><li>r/Brexit</li><li>r/Climate</li><li>r/Democrats</li><li>r/Republican</li></ol>We pulled from three subreddits of social movements r/Brexit, r/BlackLivesMatter, r/Climate, and two subreddits of political affiliations r/Republican and r/Democrats.To ensure we were only using high-quality interactions, we excluded: empty comments, comments from deleted authors, comments that were hidden for user privacy reasons, and comments containing hyperlinks. Further, in order to <a href="https://scale.com/guides/data-labeling-annotation-guide" rel="noopener noreferrer" target="_blank">annotate</a> impactful discussions in a given subreddit, we removed posts with fewer than 10 words and few comments.This dataset presents opportunities to detect (dis)agreements by leveraging context beyond just text. The set does not rely on platform-specific features, such as retweets or hashtags. The data was collected such that, for each subreddit, the resulting set of interactions formed a multi-edge, temporal graph where nodes are users, and edges represent a comment-reply interaction between two users.<h1>Challenges:</h1>Despite the preprocessing we performed to ensure <a href="https://scale.com/guides/data-labeling-annotation-guide" rel="noopener noreferrer" target="_blank">high-quality annotations</a>, the dataset annotation came with certain obstacles.We used methods that were developed in-house such as dynamic consensus to ensure high quality annotations. (For more details on dynamic consensus, take a look at our blog on <a href="https://scale.com/blog/ml-human-consensus" rel="noopener noreferrer" target="_blank">ML + Human Consensus</a>.) Annotating short online exchanges is tough; only 33% of the pairs were annotated with a full agreement among annotators. Accounting for nuanced statements, sarcasm, diverse writing genres, and specific socio-economic contexts is a tall order.<h1>Results:</h1>This research presents a challenge for NLP systems, and attempts to argue:<ul><li>Current language models struggle with messy data when it is either scraped directly from social media, or pulled from the human dialogue.</li><li>Clean and tailored datasets may not have transferable insights for online (dis)agreement detection.</li></ul>We evaluated the performance of state-of-the-art language models against our new stance detection dataset and assessed the transfer learning potential with other stance detection datasets already available.State-of-the-art language models are faring well on simple examples, but fail on more complex ones. These models tend to overly rely on superficial cues, such as the first word of the reply.Moreover, it appears that models trained on clean stance datasets behave poorly on our new dataset, which speaks to the new challenges that our dataset offers.This research presents promising avenues. Our findings highlight ways to improve language models with contextual information.We see promise in DEBAGREEMENT as it presents an opportunity to train socially aware language models.<h1>Further Research and Recommendations:</h1>We believe expanding annotations to other areas of discussion, such as consumer products reviews, investor forums or wider political forums could lead to further discoveries and insights about future consumer preferences or investor decision-making.The dataset also provides the possibility to investigate social theories, understand polarisation, study online social movements and examine how people express their opinions and change their views on social media.<h1>Read The Full Research:</h1>This work was achieved in collaboration with Oxford University, who are the first authors of the paper.The paper has been accepted to the “Datasets and Benchmarks Track” of <a href="https://neurips.cc/" rel="noopener noreferrer" target="_blank">Neurips 2021</a>. The paper details each of these issues further and is accessible on <a href="https://openreview.net/forum?id=udVUN__gFO" rel="noopener noreferrer" target="_blank">OpenReview</a>.

New Research Paves The Way to Train Socially Aware Language Models

Scale AI And Oxford University's Reddit Dataset Provides Comprehensive Online Discourse Data

At Scale, we help our customers accelerate the development of their AI applications. We do this through a variety of products and services, but here we’ll focus on one: <a href="https://scale.com/guides/data-labeling-annotation-guide" rel="noopener noreferrer" target="_blank">high-quality labels</a> for training data. We’ve developed a process for delivering model-improving data quickly and efficiently, and we highlight parts of this process below.To move fast, we align with our customers early. We strive to understand what our customers ask for and why, so that we can use the best combination of product and operational expertise to <a href="https://scale.com/guides/data-labeling-annotation-guide" rel="noopener noreferrer" target="_blank">label their data</a> with high quality - with their ML goals in mind. We take a collaborative and thoughtful approach to architecting and building data annotation pipelines. It’s one of the many reasons that customers continue to trust Scale as their AI partner.<h1>We architect data annotation pipelines that map to our customers’ ML goals</h1>At Scale, we start by understanding what our customers need to accomplish. What specifically will the AI model need to do? Is it for an <a href="https://scale.com/customers/toyota" rel="noopener noreferrer" target="_blank">Autonomous Vehicle</a> (AV) use case, perhaps predicting the action of another agent? Or is it for an application that can recognize and track objects for <a href="https://scale.com/customers/cellareye" rel="noopener noreferrer" target="_blank">inventory management</a>? From this, we can begin to understand:<ol><li>What data will the model ‘learn’ from?</li><li>What data will be used for inference, once the model is deployed in production?</li><li>What should the ideal output look like?</li><li>What determines a good (or bad) inference output?</li></ol>We’ve found that the success of an AI project is highly correlated with the quality and specificity of the training data used. This means that a broad or general training dataset won’t usually be the most efficient or quickest way to build an ML model. As a result, we collaborate with customers to set up goal-oriented annotation pipelines that map directly to their ML roadmaps. This approach helps them achieve their KPIs and project milestones.As an example, it is common for new customers to ask us to replicate a broad-purpose public dataset, such as the AV-centric <a href="https://scale.com/open-datasets/nuscenes" rel="noopener noreferrer" target="_blank">NuScenes</a>, which we released with <a href="https://www.aptiv.com/" rel="noopener noreferrer" target="_blank">Aptiv</a>. Once we deep dive on their ML roadmap however, we discover the sub-goals (such as lane planning and pedestrian intent recognition) and the desired timelines for each. We then build specialized pipelines that generate the required training data at higher quality and at faster speeds than the customer would otherwise have done themselves. In fact, our speed of operation has frequently helped customers meet key deadlines, like public conferences or demos.<img src="https://imagedelivery.net/wLbZE4_NzVVdgHc15St55g/673f8157-ef5b-4b51-a76d-ee5f8b9d0100/public" alt="Common Autonomous Vehicle ML Roadmap and Data Needs.">Scale seeks to understand our customers' timelines of ML goals and to spin up label pipelines targeted at the specific data needs for each goal.It’s important to note that few customers come to us with neatly-defined, single pipeline needs (they used to!). Today, our customers often bring multi-modal data from multiple sensors. For an AV customer, this could mean LiDAR, radar, and a combination of regular and fisheye cameras. Having a clear understanding of our customers' goals helps us be a thought partner on the types of objects or events to label across these sensors. It also helps us propose a pipeline architecture that incorporates the best product capabilities to produce a comprehensive, goal-oriented training dataset.For example, we might propose a string of dependent pipelines that weave together categorization, 3D LiDAR cuboids, and LiDAR linking. Weather attributes and 3D data defects might be caught and categorized in the categorization pipeline, vehicles might be tracked in the 3D LiDAR cuboids project, and camera-based attributes like blinker status might be generated in the linking project.<img src="https://imagedelivery.net/wLbZE4_NzVVdgHc15St55g/2e43fe86-4639-495c-3096-34676043a300/public" alt="Using specialized label pipelines to turn multi-sensor inputs into training data. ">AV customers often come to Scale with data from multiple sensors and with an idea of what labels they need. Scale's team then architects specialized label pipelines that bridge the two, producing high-quality labeled data quickly and efficiently.<h1>We test and fine tune pipeline architecture by creating a golden dataset and vetting it with our customers</h1>Following this thoughtful approach to customer alignment helps us achieve a data milestone: the creation of a golden dataset. A golden dataset is used to define what ‘good’ looks like. It is used as a reference for us, our human Taskers, and customers’ own stakeholders.Creating a golden dataset is a journey of discovery. We set out to create the golden dataset by annotating a representative sample of data in the pipeline we architected. Annotating some data always generates observations and insights about nuances in the data and requirements, which we surface to the customer along with our recommendations. We then review the resultant annotated data with our customer, tying open questions and considerations back to the customer’s ML goals.Reviewing the proposed golden dataset together with the customer is key. For example, we were working with an autonomous drone customer who needed polygon labeling of flyable space, ie where their drones could safely navigate. We found ourselves drawing pixel-perfect polygons around every tree branch, only to have the customer reject every task. Only once we debriefed live with the customer did we realize that for them, quality training data meant polygons that avoided tree pixels by a wide margin - ie, rough, simple polygons that just didn’t clip the treetops! That one collaborative sync put the project back on track with a clear path to consistent, relevant, and accurate labeled data.Attaining this mutually-vetted golden dataset is powerful. It powers our internal (human) Tasker training and assessment tools, giving both Taskers and the operations team a clear understanding of what success looks like. It also aligns our Quality Assurance (QA) team, whose job is to effectively mirror customers’ QA teams to reduce customers’ operational data-management burden.<img src="https://imagedelivery.net/wLbZE4_NzVVdgHc15St55g/ecd95dc4-c137-4a9e-4024-569dd8df3d00/public" alt="Achieving and leveraging golden data">Reviewing label samples together with customers helps establish a golden dataset, which in turn makes is possible to scale up production of high-quality labels.<h1>We engage all relevant stakeholders to ensure customers are aligned with Scale - and amongst themselves</h1>We’ve recognized that our customers are running complex initiatives involving many stakeholders on many teams, including operations, engineering, legal, and procurement. Alignment isn’t complete until Scale has aligned with each customer subteam, AND the customer subteams are internally aligned amongst themselves.Projects can fail if the customer’s operations team is grading labeled data according to a different set of standards than what the ML team expected. This can take many forms - operations might be grading too easily (for example, approving polygons that are too swaggy), or grading too harshly (for example, rejecting boxes with acceptable sizing), or just grading according to a different or outdated rubric (for example, applying an old definition of “truck” that included small pickup trucks).The result of inconsistent labeling standards is that the ML team gets training data that doesn’t improve model performance. Or, the ML team gets starved for data, even though there is data available that would have improved performance! Few things are worse than delivering weeks or months of data, with a consistent green light from the customer’s operations team - only to get a mass rejection from the ML team once it tests model performance.Scale drives stakeholder alignment by bringing the right people into the discussion, especially at steps that we know to be high risk. We also actively monitor for hints that there might be misalignment (like radio silence from some stakeholders at key milestones) and work with our customers to drive alignment internally and with Scale. We pride ourselves at being a knowledgeable partner who knows how to avoid situations that de-ccelerate the development of AI applications.<h1>Alignment is never won and done; we continue to re-assess alignment to our customers’ ML goals</h1>It takes a deliberate effort to maintain alignment. Very often, model evaluations lead customers to update their data requirements and labeling guidelines. Even when the labeling guidelines remain constant, customer data itself can change. For example, we once built a reliable high-quality pipeline to label street scenes, only to find the next batch of customer data contained scenes of people lying on a racetrack.Throughout the lifecycle of the data annotation pipeline, we continue to apply our alignment process. We continue to check in on customer ML goals, refine pipeline setup, maintain the accuracy of the golden dataset, and ensure alignment between teams on the customer, Scale, and Tasker side. This accelerates the speed with which our customers receive high-quality labeled data, which in turn accelerates their development of AI applications.To experiment with Scale’s data labelling at a small scale, sign up for <a href="https://scale.com/rapid" rel="noopener noreferrer" target="_blank">Scale Rapid</a>, the fastest way to production-quality labels, with no data minimums. To set up custom pipelines for your ML needs, <a href="https://scale.com/sales" rel="noopener noreferrer" target="_blank">talk to our team</a>.

We strive to understand what our customers ask for and why,  to label their data at high quality.

Partnering with customers to architect data pipelines

How Does Scale AI Partner with Customers to Produce High-Quality Training Data?

Every year, nearly 200,000 service members transition to civilian life. As a fighter pilot who served for over two decades in the U.S. Air Force, I know the challenges these men and women face. From adjusting to “normal” life, to mental health struggles, to determining a new career path, it can be overwhelming and difficult to navigate. One of my favorite things about Scale is our CEO’s hiring philosophy. Simply put, he says to hire people who give a shit. That philosophy has brought an amazing group of people to this company who not only care about technology, but also care about our country, our government, and our service members. As we prepare to celebrate Veterans Day, we wanted to share some of our Scale veterans’ stories to honor them and their commitment to our country, and to also offer advice for those navigating the transition from service to civilian life. <h1>What did you do in the military?</h1>Jason Schmid: I was a U.S. Army Military Intelligence Analyst, and deployed to Iraq twice. The first time was as a part of the Iraq Survey Group, determining the disposition of Iraq’s suspected weapons of mass destruction programs. On that mission, I was part of the team interrogating former high ranking Iraqi Regime officials. On my second deployment, I was embedded as a Tactical Intelligence Trainer in an Iraqi infantry battalion. Dan Tadross: I was an Air Traffic Control Officer responsible for coordinating aircraft requests and the deconfliction of airspace. In my last few years of Active Duty, I had the opportunity to serve as an Innovation Advisor to the Secretary of the Navy on how to integrate AI into Aviation Command and Control (specifically how to shorten planning cycles). I also had the privilege of being a founding member of the Department of Defense (DOD) Joint Artificial Intelligence Center (JAIC). During my two years at the JAIC, I ran a number of programs from supporting testing and evaluation to establishing Project Gargoyle for Force Protection. Brianna Hester: I was a Geospatial Analyst for 24 years. My most recent position was the Operations Superintendent of the European Partner Integration Enterprise (EPIE) where I managed a 40-person, international intelligence team operating a $5.7M Intelligence, Surveillance, Reconnaissance (ISR) coalition program for combat mission planning, execution, and overall intelligence gathering and research. I oversaw ISR mission operations and imagery exploitation of U.S. ISR platforms employed for collection of full motion video imagery in support of the DOD, NATO, and coalition forces. I deployed to Qatar and Afghanistan and also traveled to numerous other countries for my duties. Jacob Sheehan: I was a U.S. Army Special Forces Officer, also known as a Green Beret, where I deployed on combat missions to Iraq and Afghanistan and on training missions in India, Thailand, and Korea. In addition to leading operational teams, I also had an academic assignment at West Point where I taught economics courses, ran the graduate scholarship program (Rhodes, Marshall, Fulbright, etc.) and designed a two-semester critical thought curriculum. Matt Beck: I was commissioned as a Naval Officer in 2008. I spent my first two years as a Surface Warfare Officer on a destroyer. I was selected to attend SEAL training after that tour and completed training in 2012. I led teams of various sizes on deployments to Europe, Africa, South America, and the Middle East. Tyler Haley: I was a Special Operations Communicator in the U.S. Army for 10 years. I spent most of my career inside of Special Operations doing everything from tactical communications to designing and deploying data centers. I’ve spent a large portion of my career in support of overseas operations from Afghanistan, Iraq, Syria, to most of Central Africa. I was lucky enough to get to finish my career in doing what I love most, which is any and all things cloud! <h1>Why Scale?</h1>Jason Schmid: When I left government service, it was important to me to join a company that values its contribution to national security. All of the Scale senior leaders I spoke to in the interview process made it clear that Scale is dedicated to using its technology to serve the nation. I think it is hard to overstate how important the work Scale is doing for the nation, both on the commercial and federal sides of the business. Dan Tadross: One of the most challenging aspects for the U.S. government to adopt and implement AI responsibly is the lack of structured data. Scale is at the forefront of developing structured data and can do so at a volume and quality that can help catapult government efforts and ensure the competitive edge for the U.S. Brianna Hester: I was intrigued by the job description. When I opened it and looked at the requirements, I had all the requested experience so I applied. After chatting with Austin Woo about some more specifics of the position, I knew I wanted to be a part of this program. Jacob Sheehan: I was primarily drawn to Scale through my hiring manager, David Kinsella. I was headed towards working at a financial technology company in my job search, but David introduced me to Scale’s compelling vision in the AI industry and the role it could play in making the U.S. government more nimble to respond to pressing challenges. He viewed my military experience as an asset and imagined my future impact at Scale rather than solely focusing on my non-traditional business operations background. Matt Beck: I was drawn to Scale’s mission of accelerating the adoption of AI. A lot of the work we are doing on the federal team has a direct application to our nation’s defense, and I wanted to be a part of that work. Tyler Haley: I love the team dynamics of start-ups. Everyone wears different hats depending on the meeting or the time of day. This is super relatable to being on small unit teams inside of the Special Operations community. Scale, in particular, pushes the boundaries inside of technology and solves hard problems, which is exactly what I did in the U.S. military. <h1>What do you do at Scale?</h1>Jason Schmid: I am Scale’s Head of Government Relations. I work to help Congress understand AI and ML capabilities so that they can build programs and policies that will help the U.S. government accelerate the adoption of AI. Dan Tadross: I am an Engagement Manager helping to bridge the gap between U.S. government requirements and experimentation and Scale’s expertise. Brianna Hester: I am the Synthetic Aperture Radar (SAR) Imagery Intelligence Subject Matter Expert (SME). Jacob Sheehan: I’m a Business Operations and Strategy Manager that supports the federal team. My main projects include growing a compounding network of labeling experts and labeler performance management. Matt Beck: I am a Federal Deployment Strategist focused on developing solutions to support the modernization of government systems. Tyler Haley: I am the Infrastructure and Platform Lead for our federal team. I partner closely with our software engineering and security teams to help design infrastructure that works on various AWS classification regions. <h1>How was your transition from service to civilian life?</h1>Jason Schmid: I transitioned from the Army 15 years ago, and the circumstances of my transition were pretty unique. I went from being in literal combat to being a civilian in about three weeks. I was very fortunate to have a job lined up already, but it would have been much better if I had time to take advantage of all of the resources available to transitioning service members. Such a jarring transition was not easy. We now know so much more about successfully transitioning veterans back into civilian life that would have made my transition so much smoother - not just for me, but for my family as well. Dan Tadross: I transitioned off Active Duty service after 12 years during the height of the pandemic in 2020. So the transition process was stressful and challenging, but when I eventually landed at Scale my level of apprehension subsided. Brianna Hester: My transition was pretty easy, as I was ready to begin my post-military life. The hardest thing to get used to was not having to tell someone my every move. Taking PTO without telling someone the address of where I would be for the duration of the PTO was a little unnerving the first time. Jacob Sheehan: I gave myself sufficient time, but there seems to be no such thing as too much time when it comes to major life transitions, especially if you have a family to take care of! I’m glad I had the space to think about what I wanted to do next instead of having to manufacture it quickly. Transitioning during COVID was a challenge from the standpoint of young kids being around all the time, so I would recommend finding a quiet space to do your transition thinking. Matt Beck: I had a wonderful transition departing the military. I met Shands Pickett early on in the process and kept in touch as positions at Scale became available. My preparation and exploration of many different post-military careers went well and resulted in me working at Scale. Tyler Haley: Transitioning was a bit tough for me. I got out of the service in May 2020 right as the pandemic was really setting in. Transitioning from being in a room with people that you are close with to only seeing people via Zoom and Slack was difficult at first. I was accustomed to seeing my daily contributions make a real impact, but when I got out, that faded away rather quickly. The good part about transitioning was being able to be at home and start a family. My wife and I welcomed our first kiddo into the world shortly after I got out. It gave me a new purpose and perspective in life. It helped me fill the void that the military left. <h1>Can you speak to your experience as a veteran at Scale?</h1>Jason Schmid: I think one of the best qualities of Scale’s culture is a genuine curiosity and passion to solve hard problems as a team. I think anyone who has served can connect with that. Dan Tadross: One of the aspects I miss most about serving in the U.S. Marine Corps is the comradery and sense of purpose. While nothing can compare with serving in a Marine unit, Scale has a culture that shares some similarities and the team I am a part of is motivated to succeed and support the U.S. government toward the responsible adoption of AI. Brianna Hester: Scale has become my new family. I still have my military family and always will but now I have added another branch to my family tree with everyone from Scale. They may be very different families but I wouldn’t trade either of them for anything and I can’t wait to see how much we accomplish together! Jacob Sheehan: Spending time with the many veterans on the federal team has maintained some of the camaraderie I enjoyed in the U.S. Army. I’m also pleasantly surprised how genuinely curious Scaliens across all of Scale are in veteran experiences! Matt Beck: Scale is extremely supportive of its veterans. My work keeps me engaged on a mission I believe in. Furthermore, Scale is extremely flexible in letting me maintain my connection to the U.S. Navy by supporting my drill requirements. Tyler Haley: Although I am still new to the team at Scale, I felt value-added from day one. As I mentioned earlier, feeling like what you do matters is a huge point of pride for most veterans. I instantly felt as if I was a part of the team and also was surrounded by other team members that have served. At Scale, you really get the experience of being back on those small teams that have a mission and purpose. <h1>What advice would you share with veterans making the transition?</h1>Jason Schmid: Take advantage of every resource available to support your transition, whether you think it applies to you or not, because you’ve earned it. It’s normal for recently separated veterans to feel some loss of purpose. It’s important to find ways to continue serving something larger than yourself, even if it's not part of a career path. Dan Tadross: There are a lot of opportunities for those who want to find alternative ways to continue to support the U.S. government. Many of the skills I learned in the U.S. Marines continue to help me to succeed in the private sector. Those experiences and capabilities that made you successful as a service member are in demand in the private sector, so when making the transition, don’t forget the value you bring to any organization. Brianna Hester: My recommendation would be to take a break and relax for a minute before jumping right into another job, if you can. It’s a huge change going from complete structure in your everyday life to no structure, and it can be hard on some people. Enjoy a small (or even large) break and use that time as a way to prepare yourself for the great things to come. Jacob Sheehan: Spend a lot of time exploring the industry you want to work in, understanding the companies uniquely positioned in that industry, and thinking about the strengths you can bring to roles in those companies. I’ve had many veteran friends get it backwards by over-indexing on the role (title) and barely giving any thought to the company or larger industry. Matt Beck: Always seek to improve your position as you look for opportunities. Forecast your worst case scenario out as far as possible. Once you are comfortable with your worst case scenario, seek out the roles and companies that you are most excited about and get connected early. Tyler Haley: Figure out what fills your voids. Find a routine and stick to it. A lot of the service members that I speak with feel the same way. Whether that’s getting up every morning and hitting the Peloton or cooking dinner for the family every night, find something that you can do for YOU. Find a new skill or hobby and try to master it. Get out of the house and away from the keyboard. Find time in your days to step away and get lunch with another veteran. Transitioning is different for everyone. Make time to see those who understand what you are going through. If you are interested in joining our team, check out our <a href="https://scale.com/careers#jobs" rel="noopener noreferrer" target="_blank">open positions</a>.

As we prepare to celebrate Veterans Day, we wanted to share some of our Scale veterans’ stories.

Scale Team Spotlight: Veterans at Scale

<h1> </h1><h1> Why build an ML-based workflow for data labeling?</h1> Delivering high-quality data labels quickly is a challenging task. Even for humans, image and video data commonly found in self-driving or robotic applications can be compositionally challenging to parse and correctly annotate. At Scale, we use machine learning to augment human workflows to both improve the quality and speed of labeling. Since deep learning models can struggle to perform consistently across diverse data domains like self-driving car scenes, finding the optimal balance between ML automation and human supervision is necessary to ensure we maintain high-quality.Autosegment, an ML-powered tool that can accurately segment instances within a human-defined bounding box, allows us to delegate high-level reasoning about semantics and taxonomy to Taskers (human labelers) and basic pixel-level processing to an ML model. This division of labor enables us to build a general purpose and accurate ML labeling tool. Despite using a simple neural network, Autosegment outperforms prior tools.<h1> </h1><h1> The Challenge of Semantic Segmentation Labeling</h1> Semantic segmentation involves human Taskers identifying and labeling every pixel in a visual scene — a notoriously time-consuming and tedious process. Most Tasker time in this manual approach — about 95% — is consumed by time-intensive work, such as carefully tracing the outlines of objects in a scene. For example: it takes roughly one minute to trace the outline of a car and only three seconds to click on that outline and select the label “Car.” What if we could instead use machine learning to automate the most laborious parts of semantic segmentation? What if an ML model did the semantic segmentation, and the Tasker only had to select the right label?<img src="https://imagedelivery.net/wLbZE4_NzVVdgHc15St55g/72c723b3-46ae-4384-b75b-4a87b5e40b00/public" alt="Semantic Segmentation Task"> A visual scene which has been semantically segmented. Each pixel is color coded to denote the class of object it represents, i.e. Pedestrian, Road, Bike, etc. As ML engineers, we rely heavily on quantitative validation metrics, such as label accuracy and intersection-over-union (IoU), to determine the quality of our models. However, when building models that optimize for a particular business outcome — for instance, reducing the time it takes Taskers to semantically segment images — we need to think critically about which model to use and how we measure its performance.. In this example, our objective is to produce predictions that streamline the Taskers’ job. This is not necessarily the same as “best model performance.” For example, an instance segmentation model might be consistently inaccurate by just a few pixels around an object. This model might have a high IoU score but very little practical usefulness because Taskers would need to re-do every trace of the outline. Conversely, a model that can perfectly segment an object, even if it only identifies half of it, would still be extremely useful. Despite having lower IoU validation scores, it’s still an improvement to the overall workflow. Because it takes so long to trace outlines manually, having a model produce pixel-accurate partial outlines is still a significant time saver. <h1> </h1><h1> The Active Tooling Paradigm and the Autosegment Tool</h1><h1> </h1>Autosegment is a machine learning tool that segments the most salient object within a human-provided box annotation. It incorporates Tasker feedback at annotation time in a way that is flexible and iterative. When building Autosegment, we realized early on that the only way to provide flexibility and high quality was to give Taskers access to Active Tools. Active Tools let Taskers make changes to the model inputs and parameters in real time to generate the best output for their use case. This is similar to “prompt engineering,” the process by which we can coerce a neural language model like GPT-3 to generate desired text when given precise prompts. For more information, OpenAI hosts <a href="https://beta.openai.com/docs/guides/completion/prompt-design" rel="noopener noreferrer" target="_blank">excellent documentation</a> on how to design prompts for their GPT-3 model. <img src="https://imagedelivery.net/wLbZE4_NzVVdgHc15St55g/07588dc3-6a06-45e7-c61f-a51562470900/public" alt="Open-AI GPT-3 Model"> Humans working alongside models can produce better results faster than models alone Using Active Tools, we give Taskers as many options as possible to make their tasks easier. For example, they can adjust the annotation box shape and the prediction confidence threshold to configure the model to speed up their particular labeling task. <img src="https://cdn.builder.io/o/assets%2Fe0438815ba51486bbb6a202747122d4b%2Ffef154571d5846b6b87674a46a123ac5%2Fcompressed?apiKey=e0438815ba51486bbb6a202747122d4b&amp;token=fef154571d5846b6b87674a46a123ac5&amp;alt=media&amp;optimized=true" alt="Autosegment Tool Demo">A demo of the Autosegment tool. Enabling humans to freely explore the output space of the model is a key advantage of active tooling. We've already seen how this is useful. Overall, we have observed the median annotation time per scene is about 30% faster with Autosegment. We also interviewed Taskers to see how they are using the tool. While our model is trained to segment cars and motorcycles, one Tasker used the tool to segment everything. While this wasn’t a workflow we had envisioned, it halved their time per scene segmentation, from 40 minutes to 20 minutes. In a discussion we had with five Taskers, they explained that they consider Autosegment to be more accurate and efficient than previous ML solutions. Now we can simply supply robust and general online models and let Taskers optimize the rest. <h1>The Autosegment Model</h1>Autosegment is a convolutional neural network trained to segment the most salient instance within a Tasker-defined bounding box. This simple design allows us to significantly reduce our customers’ operational overhead to deploy ML, while providing pixel-perfect instance segmentations, even on previously unseen instance types. The Autosegment model does not have to be particularly complex to produce compelling results. Autosegment has no notion of different semantic types; instead, it abstractly identifies and segments based on scene composition. The model is trained to always generate a segmentation. The training data includes a few key classes like pedestrians and vehicles. The task is binary; it simply determines whether each pixel belongs to the salient object or not. We don’t train the model with negative samples. Instead, the model makes its best guess on each pixel to generate a segmentation. We define the salient instance as the object that owns the central pixel of the crop, so even if the scene is cluttered, the model can identify the target instance to segment. This binary segmentation task across instance classes and types allows the model to generalize well on new classes and taxonomies. Indeed, we can transfer not only between objects of the same class, such as cars in different countries, but between different classes altogether — for instance, moving from cars to motorcycles to street signs. This transfer can be achieved with limited retraining or, in some cases, no retraining at all, achieving zero-shot transfer. <iframe class="ql-video" frameborder="0" allowfullscreen="true" src="https://scaleai.wistia.com/embed/medias/1a4x5jij9v"></iframe> Using salient object detection on specific crops of the scene can generate even more accurate segmentation. We were able to double the edge precision score from 0.26 to 0.52 when compared to off-the-shelf full-scene semantic segmentation models. Instead of supplying a downsampled version of the whole image to a neural network, the selected region is fed to the model at full resolution. Because the input is smaller, fewer pooling operations are required to achieve a globally receptive field. As a result, there is less downsampling and blurring. Our model achieves better fidelity around the edges of instances, a crucial part of providing high-quality data annotations to our customers. Improving full-scene labeling to the same degree would require exponentially more effort and complexity. In summary, we developed Active Tooling to create an ML-based workflow capable of solving basic computer vision problems. This workflow helps our Taskers reach higher levels of quality faster than in previous workflows. Innovations like Autosegment are critical to enable the “Moore's Law of data labeling.” At Scale AI, we’re excited to push the boundaries of human-AI interaction for our customers.

Scale AI accelerates image annotation with semantic segmentation active tooling. 

Reducing image labeling time by up to 30% and doubling segmentation quality.

Faster Semantic Segmentation with Machine Learning-Assisted Workflows

AI is no longer the exclusive domain of large enterprises. It&rsquo;s entering the business mainstream&mdash;but it&rsquo;s successful only sometimes, says Salesforce President and Chief Operating Officer Bret Taylor. A thoughtful AI strategy should be customer-focused, with clearly defined business outcomes, and the key to all of this is data.
In his talk at the <a href="https://scale.com/events/transformx" target="_blank" rel="noopener noreferrer">Scale TransformX 2021 conference</a>, Taylor discussed the key considerations that businesses of all sizes should keep in mind to ensure that they delight their customers and achieve sustainable business outcomes.
<iframe class="ql-video" src="https://fast.wistia.com/embed/medias/42dyrzqs4c" frameborder="0" allowfullscreen="allowfullscreen"></iframe>
Here&rsquo;s a summary of the seven key takeaways from Taylor&rsquo;s presentation. For more details, watch the session video above.
<h3>Business Today Is All About the Data</h3>
&ldquo;If you don't have a digital business, you don't have a business,&rdquo; says Taylor.
Enterprises need to maintain constant engagement with customers, supply chains, partners, and employees, in granular detail. Each of these interactions produces data that businesses can use to create more intelligent and personalized relationships. This <a href="https://insidebigdata.com/2017/02/16/the-exponential-growth-of-data/" target="_blank" rel="noopener noreferrer">growing tsunami of data</a> requires intelligent algorithms and automation to make sense of it all. As Marc Andreessen, co-founder of venture capital firm Andreessen Horowitz, famously said, <a href="https://a16z.com/2011/08/20/why-software-is-eating-the-world/" target="_blank" rel="noopener noreferrer">&ldquo;Software is eating the world.&rdquo;</a>
Today, however, it&rsquo;s AI that has become central to everything that a company such as Salesforce does.
<h3>Build a Data Culture</h3>
AI and deep learning can&rsquo;t be the exclusive domain of technology companies. To get there, every company must strive to <a href="https://www.mckinsey.com/~/media/McKinsey/Business%20Functions/McKinsey%20Analytics/Our%20Insights/Why%20data%20culture%20matters/Why-data-culture-matters.ashx" target="_blank" rel="noopener noreferrer">develop a data culture</a> that allows employees to see and understand the flow of information moving through the business. A <a href="https://www.gartner.com/smarterwithgartner/do-you-need-a-chief-data-officer" target="_blank" rel="noopener noreferrer">chief data officer (CDO)</a> should act as a steward for all that information, without trapping data in internal silos. Instead, everyone needs to be empowered to make data-informed decisions based on objective views of customers, supply chains, and the flow of resources.
Ultimately, companies that create open data cultures will easily outcompete those that don&rsquo;t.
<h1>Ethical AI Builds Trust</h1>
Shifting toward a data culture requires building in <a href="https://www.salesforce.com/blog/what-is-ethics-by-design/" target="_blank" rel="noopener noreferrer">ethics by design</a>, in much the same way that enterprises have incorporated <a href="https://patchstack.com/security-design-principles-owasp/" target="_blank" rel="noopener noreferrer">security by design</a>.
To deliver an ethical deployment of AI, the number one value must be trust. With that in mind, Salesforce created the position of <a href="https://www.cnbc.com/2018/12/10/salesforce-hires-its-first-chief-ethical-and-humane-use-officer.html" target="_blank" rel="noopener noreferrer">chief ethical and humane use officer</a> to guide the company&rsquo;s adoption of AI.
Salesforce wanted to ensure that all of its AI is responsible, accountable, transparent, empowering, and inclusive. Taylor contends that failing to take this into account from the start inevitably leads to unintended negative consequences and general degradation of trust in technology.
Society is now holding software and AI to a higher standard. Companies that develop AI today must embrace conversations about ethics and trust, and they must embed those conversations into their development processes. Building trust also requires bringing in stakeholders who aren&rsquo;t data scientists to help shape the responsible use of these technologies.
<h3>Focus on Strategy, Not AI for AI&rsquo;s Sake</h3>
While every company needs to become an AI company to remain competitive, too many organizations are succumbing to the hype about digital transformation without having <a href="https://www.thetilt.com/content/stages-of-digital-transformation" target="_blank" rel="noopener noreferrer">clear goals</a> in mind. According to the <a href="https://www.gartner.com/en/information-technology/insights/cio-agenda" target="_blank" rel="noopener noreferrer">2021 Gartner CIO Agenda survey</a>, while 94% of CIOs are working to deploy AI, only a quarter have done so successfully.
&ldquo;Don&rsquo;t start with the technology; start with customers,&rdquo; Taylor says. Focus on their experience and what your customers need and want. Clarify the project&rsquo;s scope and look for the easy wins first. &ldquo;If AI becomes the purpose of the project, it becomes a bottomless pit of investment,&rdquo; he says.
<h3>One Shortcut to Good AI: Don&rsquo;t Build It; Buy It</h3>
Salesforce&rsquo;s <a href="https://www.salesforce.com/products/einstein/faq/" target="_blank" rel="noopener noreferrer">AI platform Einstein</a> is a prime example of Salesforce focusing on building AI technology that is customer-focused and that &ldquo;can be turned on with clicks, not code,&rdquo; says Taylor. Salesforce customer <a href="https://www.salesforce.com/blog/orvis-ai-customer-engagement/" target="_blank" rel="noopener noreferrer">Orvis uses Einstein</a> to do simple analyses of the potential engagement of its marketing messages prior to their publication.
That approach increased Orvis&rsquo; click-through rate by 22% and boosted traffic by 30%. This is one way an enterprise can deploy customer-focused experiences with the help of AI, without having to invest in the substantial skills and technology normally associated with developing it.
<h3>Align Your AI Investment with Business Priorities</h3>
A common key strategy, especially for B2C enterprises, starts with identifying use cases that increase personalization and automation, thus streamlining the customer experience. Then, identify the data and AI tools that could help create those use cases.
For its part, Orvis&rsquo; goal is to keep its support calls short, and <a href="https://www.salesforce.com/news/press-releases/2019/11/19/hey-einstein-salesforce-brings-voice-to-every-customer-experience/" target="_blank" rel="noopener noreferrer">Einstein can help by transcribing and analyzing customer questions</a> and suggesting answers to call center personnel in real time.
The lesson here: Bring in just a few machine learning experts and combine them with off-the-shelf SaaS platforms to discover unique value in AI that can be customized around your business.
<h3>Don&rsquo;t Forget the Quick Wins</h3>
Getting value from AI isn&rsquo;t science fiction, Taylor says. Simply step back and focus on the low-hanging fruit to achieve tangible business outcomes.
<h3>Learn More</h3>
To learn more about how Salesforce built its AI strategy and lessons learned,&nbsp;<a href="https://exchange.scale.com/public/videos/enterprise-ai-strategy-bret-taylor-on-lessons-learned-at-salesforce" target="_blank" rel="noopener noreferrer">watch Taylor&rsquo;s talk and view the full transcript here</a>.

Customer-first AI strategy starts with clear goals and reliable data — the key to driving real business outcomes.

AI Strategy: Data & Customer Focus

7 Elements of a Well-Thought-Out AI Strategy: Salesforce COO Bret Taylor

Dr. Li shares explores how recent AI research helps robots  perform both short-horizon and long-horizon tasks.

Dr. Fei-Fei Li On Teaching Robots Complex Tasks - TransformX 2021 Conference - Scale AI

On Teaching Robots To Perform Complex Tasks with Dr. Fei-Fei Li 

Today we are excited to welcome SiaSearch, a leading data management platform, to Scale. We acquired SiaSearch, a spin-off of the European AI venture studio Merantix, to enable us to expand Scale’s suite of tools for every AI team, broaden our footprint in European markets, and deepen the technical expertise of the company. We know that the autonomous vehicle (AV) industry is one of the best breeding grounds for AI technology. SiaSearch’s Berlin-based team brings experienced, high-caliber engineers from the European AV industry to Scale, which will help us deliver world-class AI solutions to support the complete machine learning lifecycle. The SiaSearch team will join Scale's Nucleus business unit as we grow our platform to support more data types and use cases across a wide array of industries and markets. SiaSearch’s unique interval querying engine, which is inspired by algorithms originally designed for large-scale analysis of DNA sequences, allows for lightning-fast search across massive video and LiDAR datasets and complements what Scale is building with <a href="https://studio.plasmic.app/scale.com/nucleus" rel="noopener noreferrer" target="_blank">Nucleus</a>. We are excited to weave the SiaSearch technology into the Nucleus product, accelerating development and improving the developer user experience. We see even more potential down the line to incorporate SiaSearch’s world-class team and technology throughout our entire suite of products. Merantix and the SiaSearch team have a strong network in Europe and have partnered with the world’s leading automakers, including Porsche and Volkswagen. We’re thrilled to expand Scale’s footprint to Germany and look forward to working with SiaSearch’s customers and networks to significantly increase our ability to serve European-based customers. We’re excited for what the future holds with this latest addition to our team, and we’re always looking for new opportunities. If you are interested in joining this incredible team, check out our <a href="https://scale.com/careers#jobs" rel="noopener noreferrer" target="_blank">open positions</a>. Or, if you’re interested in partnering with us, please <a href="https://scale.com/sales" rel="noopener noreferrer" target="_blank">reach out</a>.

Scale has acquired SiaSearch to further enhance Nucleus, its data management platform.

Blog