Building a natural language system for Wikipedia

Automated TestingUI Design

Overview

The Wikimedia Foundation is the non-profit media organization behind Wikipedia. Part of Wikipedia's push for wider use includes Abstract Wikipedia, which provides a language agnostic version of Wikipedia articles by utilizing natural language generation to provide content in supported languages.

Wikifunctions is being created to help facilitate process of building this multi-lingual function-as-a-service platform utlilizing open access principles found in other Wikimedia wikis.

Services Provided

This Dot Labs' engineers assisted Wikimedia by building functionality into the Wikimedia platform such as a graphical editing interface for creating, editing, and sharing functions.

Since this is a platform used by contributing site visitors, it needed to be flexible enough to handle unique types of data, and robust enough to enforce the strong type system required by Wikifunctions.

Provided a full audit of the existing code and provided structural improvement to support scalability.
Broke down components to support reusability and improve reliability by adding unit tests.
Designed and developed a highly performant VueX store to effectively support the multitude of data being handled by the application.
Enhanced the overall code structure to improve the speed of the application.

Tech-stack

Vue to provide a fast, reactive, and interactive user interface experience to users.
Vue Test Util improved components structure and reliability by adding a complete set of unit tests.
VueX is a state management tool used to abstract business logic from the component and handled the communication with the API layer.
WebdriverIO created a set of E2E testing paths to speed up development by reducing risk of regression.

This Dot Labs is a development consultancy focused on providing staff augmentation, architectural guidance, and consulting to companies.

We help implement and teach modern web best practices with technologies such as React, Angular, Vue, Web Components, GraphQL, Node, and more.

Wikimedia logo horizontal

Wikimedia Foundation is the non-profit organization supporting the free knowledge sharing platform, and world's most popular online encyclopedia: Wikipedia.

Properly Scoping and Sizing Tasks

The key to this project was ensuring that tasks were properly scoped and sized. While working on such a mission critical application, it was necessary to look closely at the specific tasks and requirements, understand how they fit into Wikimedia's overall goals, and then ensure that each piece of the application was developed in a way that could be measured and tested.

Wikimedia tasks image

Strategies for Handling Custom Data Structures

The This Dot Labs' team worked with the Wikimedia team to develop a way to handle the custom data structures in a tabular format, rather than in JSON.

This allowed developers to work on individual elements of the data object piece by piece, rather than passing portions of the object around the code.

This tabular structure was placed within Vuex, a global state management library for Vue, rather than passing pieces of data through to secondary components.

Wikimedia strategies image

Reworking Existing UI Components

With the new data structure in place, This Dot Labs' team is reworking the existing UI components (using Vue and Vuex) to utilize this new data structure.

This takes effort from the entire team to ensure that no functionality is lost. Instead, at the end, we'll have a powerful system for processing these data structures, and any changes that need to be made to them.

Wikimedia UI images

Covering Additional Data Types with Components

With the core in place, This Dot Labs' team was able to build out additional components for the UI to cover additional types of data, up to and including custom functions written in JavaScript, Python, and the platform-specific function composition.

These changes allow new components to be built much faster than and within months of development work, a fully working function editor was available for use.

Design

The This Dot Labs' team worked with the designers at Wikimedia to help them understand the project's complexity. They developed small sections of the designs quickly, thereby enabling the rest of the team and a small set of contributors to use the product.

The team actively worked to ensure the “complexity” of the product was actually provided by using a simple design (i.e. making something hard easy to use)! An agile and quick feedback loop was integral to the success of this partnership.

Before & After

Let's see how This Dot can help you

Are you looking for a development consulting firm that can provide staff augmentation, architectural guidance, and consulting to your company? Look no further than This Dot Labs. Our team of experienced developers and managers can help you implement and teach modern web best practices, and deliver your projects on time and within budget. Contact us today to learn more about our services and how we can help you achieve your development goals.

Get in touch