Web Scraping: Converting Markdown RPG Rules for RPG Maker Dataview
In this article, we will explore the process of web scraping RPG rules from freely and legally available sources, and converting them into a markdown format that can be used as a reference or database query in tools like Obsidian with plugins such as Dataview.
What is Web Scraping?
Web scraping is the process of automatically extracting large amounts of data from websites. This can be done using various tools and techniques, such as using a programming language like Python with libraries like BeautifulSoup or Scrapy. The data extracted can be used for various purposes, such as data analysis, market research, or creating a personal database.
Finding Freely and Legally Available RPG Rules
The first step in this process is to find freely and legally available RPG rules. Many RPG publishers release the rules for their games under open game licenses, which allow for the creation of derivative works. One such example is the Pathfinder Roleplaying Game (PF1E), which is released under the Open Game License. These rules can be found on the publisher's website or on other websites that host open game content.
Converting the RPG Rules to Markdown Format
Once the RPG rules have been obtained, the next step is to convert them to markdown format. Markdown is a lightweight markup language that is easy to read and write, and can be used in a variety of text editors and applications. It is also supported by many wikis and other collaborative platforms, making it an ideal format for creating a personal database of RPG rules.
To convert the RPG rules to markdown format, you can use a text editor or a specialized markdown editor. When converting the rules, it is important to maintain the original formatting as much as possible. This includes headings, lists, and any other formatting that is used in the original rules.
Using RPG Maker Dataview with the Markdown Formatted RPG Rules
Once the RPG rules have been converted to markdown format, they can be used in RPG Maker Dataview. Dataview is a plugin for Obsidian that allows you to query and display data from markdown files. This can be used to create a personal database of RPG rules that can be easily searched and filtered.
To use RPG Maker Dataview with the markdown formatted RPG rules, you will first need to create a new vault in Obsidian. Then, you can import the markdown files into the vault. Once the files have been imported, you can use the Dataview plugin to query and display the data from the files.
Web scraping and converting RPG rules to markdown format can be a powerful tool for creating a personal database of rules that can be easily searched and filtered. By using tools like RPG Maker Dataview, you can take this a step further and create a fully functional RPG rules database that can be used in your RPG projects.
References
- BeautifulSoup: https://www.crummy.com/software/BeautifulSoup/bs4/doc/
- Scrapy: https://scrapy.org/
- Pathfinder Roleplaying Game: https://paizo.com/pathfinder
- Obsidian: https://obsidian.md/
- Dataview: https://blacksmithgu.github.io/obsidian-dataview/
```python
import requests
from bs4 import BeautifulSoup
url = "https://pathfinderwrathoftherighteous.wiki.fextralife.com/Ability+Scores"
response = requests.get(url)
soup = BeautifulSoup(response.text, 'html.parser')
# Extracting the table data
table = soup.find\_all('table')[0]
rows = table.find\_all('tr')
data = []
for row in rows:
cols = row.find\_all('td')
cols = [col.text.strip() for col in cols]
data.append([col for col in cols if col])
# Printing the data
print(data)
This is an example of how you can use Python and BeautifulSoup to scrape the ability scores table from the Pathfinder Wrath of the Righteous wiki. The data is then stored in a list of lists, where each inner list represents a row in the table.
You can then use this data to create a markdown table, or use it in any other way you see fit.