• Zilliz Cloud is utilized to drive AI models for product recommendation engines, semantic text search, targeted advertising, risk management, and fraud protection.
• Zilliz Cloud can offer the foundation for a ChatGPT/VectorDB/Prompts-as-Code technological stack that, according to the business, enables LLMs to vastly scale out their expertise by gaining access to multiple other data sources.

Zilliz Inc., a startup specializing in vector databases, has announced that its newly upgraded Zilliz Cloud solution is what artificial intelligence practitioners need to prevent hallucinations.

Large language models, such as ChatGPT, have captivated the public’s curiosity because of their amazing ability to generate human-like replies to virtually any query. Nevertheless, Zilliz observes that these models are far from ideal, with one of their significant flaws being that they frequently generate answers in the absence of accurate data. The AI industry refers to this phenomenon as “hallucination,” It may be highly dangerous in some scenarios, such as when AI is employed to answer customer service concerns.

Zilliz thinks it can avoid these hallucinations. It quotes OpenAI LP, the developer of ChatGPT, noting that these falsified replies may be reduced by providing LLMs with external sources of domain-specific data, which Zilliz Cloud can assist in.

Zilliz Cloud is a vector database supporting AI applications built on the open-source Milvus project. These models often translate unstructured data such as texts, videos, and user actions into vectors, which are complicated numerical sequences. So, it is frequently necessary to determine which vectors are closest to or most comparable to others.

A specialized vector database is crucial while sorting and ranking a lot of vectors. Conventional databases are intended to store tables and documents, making them inefficient for machine learning. Zilliz Cloud is unique because it can dynamically alter and index millions of vectors to respond to the usual queries posed to AI models.

Zilliz Cloud drives AI models for product recommendation engines, targeted advertising, semantic text search, risk management, and fraud protection. Nonetheless, its vector-native architecture makes it perfect for LLMs as well.

By utilizing OpenAI plugins to connect to ChatGPT, Zilliz Cloud can offer the foundation for a ChatGPT/VectorDB/Prompts-as-Code technological stack that, according to the business, enables LLMs to vastly scale out their expertise by gaining access to multiple other data sources. With greater understanding, hallucinations should become far less frequent.

Charles Xie, Chief Executive, said that if AI is supposed to fulfill its potential, it needs to become more trustworthy. He further said, “Hallucinations or wrong answers erode that trust. With the billion-vector performance of Zilliz, we can help address that by expanding context and data retrieval.”

• Cloud Spanner is a fully managed relational database service that offers global scalability, high consistency, and high availability.
• Developers may now evaluate Spanner at no cost, taking advantage of several guided tutorials and a sample database.

Google Cloud announced significant changes to several of its major database systems, including a free trial for those looking to test Cloud Spanner. It also launched Datastream, a new service for BigQuery that makes it easier to gain real-time insights.

The majority of the announcements are focused on expanding the possibilities of cloud databases. Andi Gutmans, Vice President and General Manager of the database at Google, said, “We’ve passed the tipping point for databases in the cloud,” noting that Gartner Inc. says more than half of database usage is now in the cloud. A lot of that is fueled by transformative applications.”

Cloud Spanner is a fully managed relational database service that offers global scalability, high consistency, and high availability. It is a commercial version of the relational database that Google uses internally to run its consumer services, such as Google Search, Gmail, etc. The system uses conventional SQL syntax for queries and includes service-level agreements that guarantee a maximum uptime of 99.999%. This equates to less than one hour of downtime annually.

Sailesh Krishnamurthy, Vice President of engineering for databases at Google Cloud, stated that he anticipates many developers would desire to explore and test the possibilities of Spanner. However, many have been unable to do so due to the cost of the service, which makes basic experimental programs unaffordable. To address this issue, Google has announced the availability of free trial instances for Spanner, allowing users to test the service without any expense.

Developers may now evaluate Spanner at no cost, taking advantage of several guided tutorials and a sample database. According to Krishnamurthy, the trial instances come with 10 GB of storage for up to 90 days. Users may unlock Spanner’s full capabilities at any moment during the trial by purchasing a subscription to get limitless scale and multi-region deployments.

Gutmans said the business experienced a 10-fold increase in Spanner demand after it announced more granular instance sizes. “We hope the free trial will help,” he added.

The preview availability of fine-grained access control is the second upgrade for Spanner. Users may now approve access to Spanner data at the table and column level. The database already offers identity and access management controls, but the fine-grained access is intended to make it easier to safeguard transactional data and guarantee adequate security when allowing access to particular portions of the database. For instance, according to Gutmans, if the database contains personally identifiable information such as Social Security numbers, such information can be concealed from developers creating applications using Spanner.

Datastream for BigQuery, Google’s serverless cloud data warehouse, will also be available in preview. Datastream is a new technology that enables users to easily enable real-time insights in BigQuery by following a few simple steps.

According to Google, the agentless solution replicates every event reliably as it happens and replicates data from operational database sources directly into BigQuery. Then, users may set up an extract, load and transform pipeline for low-latency data replication that enables real-time insights.

Klook Travel Technology Ltd’s senior data manager, Stacy Zhu, said her organization has been testing Datastream for BigQuery for quite some time to make data-driven business choices.

Stacy Zhu said, “Prior to adopting Datastream, we had a team of data engineers dedicated to the task of ingesting data into BigQuery, and we spent a lot of time and effort making sure that the data was accurate. With Datastream, our data analysts can have accurate data readily available to them in BigQuery with a simple click. We enjoy Datastream’s ease of use, and its performance helps us achieve large scale ELT data processing.”

Sources for Datastream include MySQL, PostgreSQL, AlloyDB, and Oracle databases, which can be hosted on-premises or on Google Cloud’s infrastructure.

According to Krishnamurthy, many businesses are seeking to transition from historical databases to PostgreSQL to prevent vendor lock-in, shorten the learning curve for development teams, and decrease expenses. Today, Krishnamurthy announced that Google Cloud’s Database Migration Service allows migrations from PostgreSQL databases to AlloyDB, a PostgreSQL-compatible service that is reportedly four times quicker when performing transactional workloads. With DMS support, customers can transfer PostgreSQL workloads to AlloyDB with only a few clicks.

Brazil’s CURA Grupo, a medical diagnostics giant, made this decision because it was experiencing growing issues with its on-premises PostgreSQL database. With the necessity to preserve the results of more than 30,000 medical tests performed daily, CURA Grupo determined that cloud storage would be more practical. Thanks to DMS, the database migration from PostgreSQL to AlloyDB required only 20 minutes of downtime.

• A report by New Vantage Partners has revealed that 97% of firms are actively investing in data, but the goal of being genuinely data-driven remains elusive.
• Dataddo delivers patented anomaly detection technology and automated format harmonization to prepare unstructured data dashboards for business teams.

Dataddo, a no-code data integration platform, has unveiled a free plan to allow corporate users of all levels access and begin working with data from diverse sources.

A report by New Vantage Partners has revealed that 97% of firms are actively investing in data, but the goal of being genuinely data-driven remains elusive. Just 26.5% of the respondents said they had created a data-driven organization and less than half claimed to be competing on data and analytics.

Another report by Accenture also discovered that 74% of corporate users “felt overwhelmed or unpleasant while working with data.”

In truth, there might be various causes for these obstacles, but a majority of the respondents cited the lack of an adequate data culture as the greatest hurdle.

The New Vantage Partners survey said, “There continues to be a strong consensus that cultural challenges — slow adaptability to change, reengineering of business processes, education of staff, requirements for data literacy, organizational alignment to support business objectives — represent the major impediments organizations face in their efforts to become data-driven. For the fourth consecutive year, over 90% of executives — 91.9% in 2022 — point to culture as the greatest impediment to achieving this business outcome.”

Dataddo free data integration offering

To set the ball rolling, Dataddo has developed a free tier of its software to address this difficulty and foster a better data culture within enterprises.

With Dataddo’s offering, enterprise users may access any 10 of Dataddo’s 200+ connections and automate the synchronization of marketing, sales, financial, and other cloud data to up to three dashboarding apps and Google Sheets weekly.

In this manner, firms may begin data integration on a lesser scale and familiarize/educate staff without investing in a commercial offering immediately. Before implementing it entirely in a data warehouse, the solution may also be used to evaluate the validity of a data model in beta form.

Petr Nemeth, founder and CEO of Dataddo, said, “Once businesses outgrow our free plan, there is no limit to how much further they can scale up integration (with paid plans). Whether they are just starting to analyze data in Google Sheets or ready to integrate our API into their own app, Dataddo supports companies at every stage of data maturity.”

No extraction or time limits

While the free plan puts restrictions around connections and target dashboarding apps, it does not cap extraction limits (users can sync any volume of data) or put any time constraints on service usage. Dataddo claims to be the first data integration platform to provide this functionality.

The firm also provides two subscription programs for those who want to go to the next level: Data to Dashboards and Data Anywhere. The former is a fully-featured version of the free plan that grants customers access to all 200+ connectors to sync data with more than three visualization tools. On the other hand, the latter enables subscribers to transmit data to warehouses, between warehouses, and back into applications (reverse ETL). In addition, it provides a growing collection of free templates for popular dashboarding software, allowing professionals of any degree of data expertise to begin data analysis quickly.

Dataddo delivers patented anomaly detection technology and automated format harmonization to prepare unstructured data dashboards for business teams. As for more advanced insights, the technology enables users to integrate datasets from several systems before exporting them to a dashboard.

Petr Nemeth stated, “The Dataddo platform is designed for non-technical business professionals. But it’s also incredibly friendly to engineers because it allows them to tap into low-level platform operations via an API.”

Other announcements

In addition to the free plan, Dataddo announced the release of reverse ETL capability and a headless data integration solution that enables businesses to develop their data products on top of the unified Dataddo API.

• Google Play Store included a new data privacy section requiring developers to declare the information their applications gather.
• Google provides developers complete flexibility over what they want to share with users.

The Google Play Store introduced a new data privacy section at the beginning of this year that relies on app developers to reveal the information collected by their apps. Google will no longer display an ascertained list of permissions it automatically contains from each app, giving developers complete control over what they choose (or don’t choose) to disclose to users.

When Google announced the new data privacy policies almost a year ago, the firm made it clear that its system would rely on the information given by developers. A support website by Google mentions that developers have until the 20th of July to fill out a data privacy form for their applications. The page also notes that developers “alone” are responsible for making “full and correct disclosures” for their apps.

Apple’s privacy policy

Apple’s App Store has a similar regulation in place for its privacy “nutrition” labels. It also asks developers to provide “self-reported summaries” regarding the privacy policies of their products. Both these requirements are discussed more below. Apple, like Google, trusts developers to provide truthful information about the data their apps collect. But according to a report published by a leading newspaper, this aspect is often “misleading or flat-out inaccurate.”

Though Google has not specified any plan to replace the automatically generated app permissions with the data privacy section, it seems it swapped it quietly.

Several Screenshots have been shared on Twitter comparing one app listing with the old “permissions” section and one with merely “data safety.” The same was noticed after comparing an archived version of TikTok’s Google Play Store listing from 2021 and the current version.

The default store on most Android devices is Google Play. Hence, the store is widely used by Android users to download new apps and games and to update them.

Till date, Google Play displayed permissions needed by an application to function. A base set of permissions are granted to all applications that are not highlighted; it goes beyond what needs to be specified in the application’s manifest.

While browsing apps or games on Google Play, one may discover that the permissions listing is not available anymore. It appears that Google decided that the new data safety listing is sufficient. Google play app permissions.

As Google has made it mandatory, publishers are now needed to give data safety information. It is up for the publisher to fill out the info. Neither the Google Play application nor the Google Play store website list permissions anymore.

Google has not provided any reason as to why it decided to eliminate the option to view application and game permissions on its Android store. There’s a possibility though that Google thought that the new Data Safety listing was enough or that permissions could scare users.

However, options to display permissions exist. A better option, according to a Twitter user is Aurora, an open-source alternative to Google’s Play Store, shows permissions before downloading an app, which he recommends.

That being said, it would be much more logical for Google to provide both the app permissions and the data privacy area.

The users will be able to validate that the developer-reported permissions are compatible with Google’s results if it displays both the app permissions and the data privacy section. If Google displays both, the users will be able to compare both.

Meanwhile, Google explained, “Google Play reviews apps across all policy requirements; however, we cannot make determinations on behalf of the developers of how they handle user data. Only you possess all the information required to complete the Data safety form.” Google says it will take “appropriate action if it finds any discrepancies between developers’ reported information and the app itself.”

And guess what? We found a way to save it by switching to a decentralized storage protocol.

What is decentralized storage protocol?

Decentralized storage protocols are designed using blockchain technology. It provides an incentive system to enable people to store data on multiple network nodes. These protocols introduce a tokenomics model to realize distributed and piecewise data storage on multiple network nodes. The two major categories of decentralized storage protocols are:

1. One category storage as arithmetic power and users mine by providing hard disk space. The consensus mechanism is benchmarked against Bitcoin-type projects, and the representative project is called Siacoin.

2. The other is storage as a service, enabling users to store data as arithmetic power.

In simple words,

One of the key components of a decentralized web is its storage system. The data is distributed in different chunks among various peer-to-peer network nodes. Several distributed file-sharing systems such as Napster and Bit torrent have been tried and tested before. However, these were not designed to be built-in infrastructure. The advantages of storing data in a decentralized environment are similar to those of a decentralized web.

Why get decentralized storage protocols within the organization?

It is recommended to take decentralized storage protocols for some of the valid reasons mentioned below:

1. Security –

Each copy of data is encrypted and stored in its own unique and secure way. Only the private key can decrypt it and view it in digital currency. Data is stored in a decentralized manner, which means that only the parts of the data are affected by hackers.

2. Reliability –

Blockchain is a digital ledger that enables transactions to be distributed without intermediaries. Its decentralized nature allows storage protocols to be used for storing data. A cloud storage protocol is a way to store data that uses decentralized storage and decentralization characteristics.

In the decentralized storage protocol, all the blocks are verified. Workers should verify any deposited data, that is to prevent data tampering.

A distributed storage system achieves load balancing, preventing traffic from overwhelming a single location. Since users store multiple copies of their data in different areas, they do not suffer data loss if their machines fail or stop working.

3. Honestly

Since there will be so many decentralized nodes, the market will be more open and competitive. The price of data storage can vary widely depending on the storage platform used and the complexity of the storage requirements.

For the next thing to do, you must look up some popular decentralized storage systems within your organization.

Known some popular decentralized storage systems

1. InterPlanetary File System (IPFS)

IPFS is a distributed file system that can replace HTTP. It is similar to HTTP in that it sends and receives files over HTTP. IPFS is a vital component of Skeps’ decentralized architecture. It helps secure the transfer of files between nodes.

2. Storj

The Storj network is a distributed object store that enables users to store and distribute data globally. In short, it is a private, decentralized, affordable, and secure cloud object storage for developers. It enables any computer running its software to rent unused hard drive space to individuals looking to store data. Storj is an subsitute to cloud storage platforms like those offered by Google and Amazon. However, Storj depends on software and the network of computers to manage its data storage. Significant components of Storj are Satellites, Storage Nodes, and Uplinks.

3. OrbitDB

OrbitDB is a distributed, serverless, and peer-to-to-peer database. It uses IPFS to store and distribute its data and IFPS Pubsub to sync the database with peers automatically. The data in OrbitDB should be stored in various ways, such as sharded or kept in its original state. It supports multiple data models, such as keyvalue, log, feed, counter, and documents. OrbitDB databases are consistent, achieved with Conflict-Free Replicated Data Type (CRDTs) for conflict-free database, making it an excellent choice for decentralized Apps (dApps), offline-first web applications, and blockchain applications.

4. Skeps

At Skeps, they build a distributed database called ChainWolfDB, which OrbitDB inspires. Its goal is to provide a distributed database with consistent and securely stored data. It uses encryption to transfer data and supports transactions that help keep all the nodes involved in the data safe.

5. GUN

GUN is a real-time, decentralized, offline-first graph database engine that is lightweight and powerful. The GUN is a distributed database system that can sync data between connected nodes. Its goal is to provide consistent and real-time updates.

Wrap-up

The flexible data storage model is used for storing data in various configurations. It can store relational data (MSSQL or MySQL), tree-structured document orientation (MongoDB), or a graph.

In the Web 3.0 era, where data ownership and security are more critical, reliability, safety, and scalability will profoundly impact companies.

The curtain has just turned on the Web 3.0 revolution, and decentralized storage still has a long way to go in infrastructure, talent, infrastructure, and funding.

Although decentralized storage is still in its early stages, the national policies will also influence the development of the industry. Despite the various national policies and market trends, centralized storage platforms are still the mainstream method for storage.

We are on the verge of rapid technological advancements and are still waiting for decentralized storage protocols to exist.

For more such content download our latest whitepapers on storage here.